Metadata-Version: 2.4
Name: zodbpickle
Version: 4.3
Summary: Fork of Python 3 pickle module
Author-email: Python and Zope Foundation and contributors <zope-dev@zope.dev>
Maintainer-email: Plone Foundation and contributors <zope-dev@zope.dev>
License-Expression: ZPL-2.1
Project-URL: Issues, https://github.com/zopefoundation/zodbpickle/issues
Project-URL: Source, https://github.com/zopefoundation/zodbpickle
Project-URL: Changelog, https://github.com/zopefoundation/zodbpickle/blob/master/CHANGES.rst
Keywords: zodb,pickle
Classifier: Development Status :: 5 - Production/Stable
Classifier: Framework :: ZODB
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/x-rst
License-File: LICENSE.txt
Provides-Extra: docs
Requires-Dist: Sphinx; extra == "docs"
Requires-Dist: furo; extra == "docs"
Provides-Extra: test
Requires-Dist: zope.testrunner>=6.4; extra == "test"
Dynamic: license-file

``zodbpickle`` README
=====================

.. image:: https://github.com/zopefoundation/zodbpickle/actions/workflows/tests.yml/badge.svg
        :target: https://github.com/zopefoundation/zodbpickle/actions/workflows/tests.yml

.. image:: https://coveralls.io/repos/github/zopefoundation/zodbpickle/badge.svg
   :target: https://coveralls.io/github/zopefoundation/zodbpickle
   :alt: Coverage status

.. image:: https://img.shields.io/pypi/v/zodbpickle.svg
        :target: https://pypi.python.org/pypi/zodbpickle
        :alt: PyPI

.. image:: https://img.shields.io/pypi/pyversions/zodbpickle.svg
        :target: https://pypi.python.org/pypi/zodbpickle
        :alt: Python versions

This package presents a uniform pickling interface for ZODB:

- Under Python2, this package forks both Python 2.7's ``pickle`` and
  ``cPickle`` modules, adding support for the ``protocol 3`` opcodes.
  It also provides a new subclass of ``bytes``, ``zodbpickle.binary``,
  which Python2 applications can use to pickle binary values such that
  they will be unpickled as ``bytes`` under Py3k.

- Under Py3k, this package forks the ``pickle`` module (and the supporting
  C extension) from both Python 3.2 and Python 3.3.  The fork add support
  for the ``noload`` operations used by ZODB.

Caution
-------

``zodbpickle`` relies on Python's ``pickle`` module.
The ``pickle`` module is not intended to be secure against erroneous or
maliciously constructed data. Never unpickle data received from an
untrusted or unauthenticated source as arbitrary code might be executed.

Also see https://docs.python.org/3.6/library/pickle.html

General Usage
-------------

To get compatibility between Python 2 and 3 pickling, replace::

    import pickle

by::

    from zodbpickle import pickle

This provides compatibility, but has the effect that you get the fast implementation
in Python 3, while Python 2 uses the slow version.

To get a more deterministic choice of the implementation, use one of::

    from zodbpickle import fastpickle # always C
    from zodbpickle import slowpickle # always Python

Both modules can co-exist which is helpful for comparison.

But there is a bit more to consider, so please read on!

Loading/Storing Python 2 Strings
--------------------------------

In all their wisdom, the Python developers have decided that Python 2 ``str``
instances should be loaded as Python 3 ``str`` objects (i.e. unicode
strings). Patches were proposed in Python `issue 6784`__ but were never
applied. This code base contains those patches.

.. __: http://bugs.python.org/issue6784

Example 1: Loading Python 2 pickles on Python 3 ::

    $ python2
    >>> import pickle
    >>> pickle.dumps('\xff', protocol=0)
    "S'\\xff'\np0\n."
    >>> pickle.dumps('\xff', protocol=1)
    'U\x01\xffq\x00.'
    >>> pickle.dumps('\xff', protocol=2)
    '\x80\x02U\x01\xffq\x00.'

    $ python3
    >>> from zodbpickle import pickle
    >>> pickle.loads(b"S'\\xff'\np0\n.", encoding='bytes')
    b'\xff'
    >>> pickle.loads(b'U\x01\xffq\x00.', encoding='bytes')
    b'\xff'
    >>> pickle.loads(b'\x80\x02U\x01\xffq\x00.', encoding='bytes')
    b'\xff'

Example 2: Loading Python 3 pickles on Python 2 ::

    $ python3
    >>> from zodbpickle import pickle
    >>> pickle.dumps(b"\xff", protocol=0)
    b'c_codecs\nencode\np0\n(V\xff\np1\nVlatin1\np2\ntp3\nRp4\n.'
    >>> pickle.dumps(b"\xff", protocol=1)
    b'c_codecs\nencode\nq\x00(X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02tq\x03Rq\x04.'
    >>> pickle.dumps(b"\xff", protocol=2)
    b'\x80\x02c_codecs\nencode\nq\x00X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02\x86q\x03Rq\x04.'

    $ python2
    >>> import pickle
    >>> pickle.loads('c_codecs\nencode\np0\n(V\xff\np1\nVlatin1\np2\ntp3\nRp4\n.')
    '\xff'
    >>> pickle.loads('c_codecs\nencode\nq\x00(X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02tq\x03Rq\x04.')
    '\xff'
    >>> pickle.loads('\x80\x02c_codecs\nencode\nq\x00X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02\x86q\x03Rq\x04.')
    '\xff'

Example 3: everything breaks down ::

    $ python2
    >>> class Foo(object):
    ...     def __init__(self):
    ...         self.x = 'hello'
    ...
    >>> import pickle
    >>> pickle.dumps(Foo(), protocol=0)
    "ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'x'\np6\nS'hello'\np7\nsb."
    >>> pickle.dumps(Foo(), protocol=1)
    'ccopy_reg\n_reconstructor\nq\x00(c__main__\nFoo\nq\x01c__builtin__\nobject\nq\x02Ntq\x03Rq\x04}q\x05U\x01xq\x06U\x05helloq\x07sb.'
    >>> pickle.dumps(Foo(), protocol=2)
    '\x80\x02c__main__\nFoo\nq\x00)\x81q\x01}q\x02U\x01xq\x03U\x05helloq\x04sb.'

    $ python3
    >>> from zodbpickle import pickle
    >>> class Foo(object): pass
    ...
    >>> foo = pickle.loads("ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'x'\np6\nS'hello'\np7\nsb.", encoding='bytes')
    >>> foo.x
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'Foo' object has no attribute 'x'

wait what? ::

    >>> foo.__dict__
    {b'x': b'hello'}

oooh.  So we use ``encoding='ASCII'`` (the default) and ``errors='bytes'`` and
hope it works::

    >>> foo = pickle.loads("ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'x'\np6\nS'hello'\np7\nsb.", errors='bytes')
    >>> foo.x
    'hello'

falling back to bytes if necessary ::

    >>> pickle.loads(b'\x80\x02U\x01\xffq\x00.', errors='bytes')
    b'\xff'


Support for ``noload()``
------------------------

The ZODB uses `cPickle`'s ``noload()`` method to retrieve all persistent
references from a pickle without loading any objects. This feature was removed
from Python 3's pickle. Unfortuantely, this unnecessarily fills the pickle
cache.

This module provides a ``noload()`` method again.
