Python packaging in a nutshell

Why I should use it?

1. It is the most convenient way to spread and share your work
2. PEP 20There should be one-- and preferably only one --obvious way to do it.
3. As a package maintainer you have to use it (and yes it is old ~ 1998)
4. Basically it's a way to distribute software

The Python Package Index is a repository of software for the Python programming language. 45876

1. Fast (Fastly CDN)
2. Reliable :)
3. Secure (?)

Official guidelines?

1. https://docs.python.org/3/distutils/packageindex.html
2. http://python-packaging-user-guide.readthedocs.org/en/latest/
3. setuptools vs distutils?
4. pip vs easy_install?
5. ...

Python Packaging Authority (PyPA)

Minimal structure

        
          ➜  development $ tree -L 2 VotePackage
          VotePackage
          ├── MANIFEST
          ├── README.txt
          ├── setup.py
          └── votepackage
              └── __init__.py
          2 directories, 6 files
        
      

Minimal setup.py data

        
          from distutils.core import setup
          setup(
              name='VotePackage',
              version='0.1dev',
              author='Vasyl Dizhak',
              author_email='vasyl.dizhak@djangostars.com',
              description='Centralized system for python package voting',
              packages=['votepackage',],
              license='Simplified BSD License',
              long_description=open('README.txt').read(),
          )
        
      

Minimal setup.py data: name

        
          ...
          name='VotePackage'
          ...
        
      

        
          http://pypi.python.org/pypi/<name>
        
      

Minimal setup.py data: packages

packages=['votepackage', 'votepackage.tests']
        
      

                package_dir = {'': 'src/lib'}
            

Minimal setup.py data: package data

setup(...,
      package_data={'votepackage': ['stats/*.json']},
      ...
     )
        
      

Installing Additional Files

setup(...,
      data_files=[
          ('/etc/init.d', ['pypivote-server']),
          ]
      ...
     )
        
      

Minimal setup.py data: version

        
 >>> from distutils.version import LooseVersion as V
 >>> v1 = V('1.0dev')
 >>> v2 = V('1.0.9alpha')
 >>> v2 > v1
 >>> False
        
      

        
$ pip install -U name
        
      

Minimal setup.py data: author

        
  author='Vasyl Dizhak',
  author_email='vasyl.dizhak@djangostars.com',
  
  
  maintainer='Vasyl Dizhak',
  maintainer_email='vasyl.dizhak@djangostars.com',
 
        
      

Minimal setup.py data: description

long_description=open('README.md').read(),
        
      

Minimal setup.py data: MANIFEST.in

include LICENSE
include README.rst
recursive-include votepackage/static *
recursive-include votepackage/templates *
        
      

setup: requirements

        install_requires = [
            'requests',
            'pycli'
        ],

        
      

Rely on platform or python version

        if sys.version_info[:2] in ((2, 6), (3, 1)):
            # argparse has been added in Python 3.2 / 2.7
            requirements.append('argparse>=1.2.1')
        if 'win32' in str(sys.platform).lower():
            # Terminal colors for Windows
            requirements.append('colorama>=0.2.4')
        
      

Classifiers

        classifiers=[
            'Development Status :: 1 - Planning',
            'Intended Audience :: Developers',
            'Environment :: Console',
            'Programming Language :: Python :: 2.5',
            'Programming Language :: Python :: 2.6',
            'Programming Language :: Python :: 2.7',
            'Programming Language :: Python :: 3.3'
            'Framework :: Django'
        ]
        python setup.py register --list-classifiers
        
      

Console scripts

        setup(...,
            scripts=['django/bin/django-admin.py']
         )

        entry_points={
            'console_scripts': [
                'pypivote=votepackage:main',
           ],
        
      

What have been done?

1. Build distributition VotePackage-0.1dev.tar.gz and upload it to PYPI
2. You could build a package for your OS bdist_wininst, bdist_rpm etc
3. You can use --repository=url for uploading package to your own cheeseshop

Tox

Use tox to automate package testing against different python versions

      
        import sys
        from setuptools.command.test import test as TestCommand
        class Tox(TestCommand):
            def finalize_options(self):
                TestCommand.finalize_options(self)
                self.test_args = []
                self.test_suite = True
            def run_tests(self):
                #import here, cause outside the eggs aren't loaded
                import tox
                errno = tox.cmdline(self.test_args)
                sys.exit(errno)
      
    

Tox.ini & setup.py

      
        # content of: tox.ini , put in same dir as setup.py
        [tox]
        envlist = py25,py26,py27,py33
        [testenv]
        deps=pytest       # install pytest in the venvs
        commands=py.test  # or 'nosetests' or ...

      
    

      
        # setup.py
        ...
        tests_require=['tox'], #install tox package
        cmdclass = {'test': Tox},
      
      
    

Test your package

      
        $ python setup.py test
        ...
        py27 inst-nodeps: /Users/rootart/development/
          VotePackage/.tox/dist/VotePackage-0.1dev.zip
        py27 runtests: commands[0]
        ...
        py25: commands succeeded
        py26: commands succeeded
        py27: commands succeeded
        py33: commands succeeded
        congratulations :)
      
    

Use CI system for constant builds

• https://travis-ci.org/
• https://www.shiningpanda-ci.com/
• http://jenkins-ci.org/
• http://trac.buildbot.net/
• ...

PIP & virtualenv

      
        $ virtualenv env
        $ pip install/uninstall package
        $ pip freeze > requirements-dev.txt
        $ pip install -r requirements-dev.txt
        $ pip install ... -M (--mirrors) #PEP-381
        $ pip install -e git/hg/svn
        $ pip list -o #get outdated packages
      
    

PIP useful configuration options

      
        # $HOME/.pip/pip.conf
        [global]
        timeout = 60
        index-url = http://local.cheeseshop/
        download_cache = ~/.cache/pip
      
    

The Wheel Binary Package Format

      
python setup.py bdist_wheel --universal
          ....
pip wheel --wheel-dir=/local/wheels -r requirements.txt
pip install --no-index --find-links=/local/wheels \
          -r requirements.txt
      
    

Twine is a utility for interacting with PyPI

twine upload dist/*

Future of python packaging

1. standalone setup tool that install and uninstall packages
2. metadata will be available outside of Python file in setup.cfg
3. already in Python 3 as a setuptools
4. pre-releases
5. PEP 426

Some of the distutils2 features

      
        Requires-External: C
        Requires-External: libpng (>=1.5) #PIL :)
        
        Requires-Dist package; python_version == '2.5'
        
        Obsoletes-Dist: OtherProject (<3.0)
      
    

Say hello to PIP (PEP 453)

Python Warehouse

1. KISS
2. Follow a standards
3. Use best practices of virtualenv/pip
4. Read PEPs and lookout for new changes in distutils2
5. Share your experience
6. http://packaging.python.org