InfoSec Forums InfoSec Forums Programming awesome-python: A curated list of awesome Python frameworks, libraries, software and resources

This topic contains 0 replies, has 1 voice, and was last updated by do son do son 4 weeks, 1 day ago.

  • Author
    Posts
  • #4534
    do son
    do son
    Moderator
    • Topics:202
    • Comments:51
    • Super Member
    • ★★★★

    A curated list of awesome Python frameworks, libraries, software and resources.

    Inspired by awesome-php.


    Admin Panels

    Libraries for administrative interfaces.

    • Ajenti – The admin panel your servers deserve.
    • django-suit – Alternative Django Admin-Interface (free only for Non-commercial use).
    • django-xadmin – Drop-in replacement of Django admin comes with lots of goodies.
    • flask-admin – Simple and extensible administrative interface framework for Flask.
    • flower – Real-time monitor and web admin for Celery.
    • Grappelli – A jazzy skin for the Django Admin-Interface.
    • Wooey – A Django app which creates automatic web UIs for Python scripts.

    Algorithms and Design Patterns

    Python implementation of algorithms and design patterns.

    • algorithms – A module of algorithms for Python.
    • PyPattyrn – A simple yet effective library for implementing common design patterns.
    • python-patterns – A collection of design patterns in Python.
    • sortedcontainers – Fast, pure-Python implementation of SortedList, SortedDict, and SortedSet types.

    Anti-spam

    Libraries for fighting spam.

    Asset Management

    Tools for managing, compressing and minifying website assets.

    • django-compressor – Compresses linked and inline JavaScript or CSS into a single cached file.
    • django-pipeline – An asset packaging library for Django.
    • django-storages – A collection of custom storage back ends for Django.
    • fanstatic – Packages, optimizes, and serves static file dependencies as Python packages.
    • fileconveyor – A daemon to detect and sync files to CDNs, S3 and FTP.
    • flask-assets – Helps you integrate webassets into your Flask app.
    • jinja-assets-compressor – A Jinja extension to compile and compress your assets.
    • webassets – Bundles, optimizes, and manages unique cache-busting URLs for static resources.

    Audio

    Libraries for manipulating audio.

    • audiolazy – Expressive Digital Signal Processing (DSP) package for Python.
    • audioread – Cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding.
    • beets – A music library manager and MusicBrainz tagger.
    • dejavu – Audio fingerprinting and recognition.
    • django-elastic-transcoder – Django + Amazon Elastic Transcoder.
    • eyeD3 – A tool for working with audio files, specifically MP3 files containing ID3 metadata.
    • id3reader – A Python module for reading MP3 meta data.
    • m3u8 – A module for parsing m3u8 file.
    • mingus – An advanced music theory and notation package with MIDI file and playback support.
    • mutagen – A Python module to handle audio metadata.
    • pydub – Manipulate audio with a simple and easy high level interface.
    • pyechonest – Python client for the Echo Nest API.
    • talkbox – A Python library for speech/signal processing.
    • TimeSide – Open web audio processing framework.
    • tinytag – A library for reading music meta data of MP3, OGG, FLAC and Wave files.

    Authentication

    Libraries for implementing authentications schemes.

    • OAuth
      • Authomatic – Simple but powerful framework agnostic authentication/authorization client.
      • django-allauth – Authentication app for Django that “just works.”
      • django-oauth-toolkit – OAuth 2 goodies for Django.
      • Flask-OAuthlib – OAuth 1.0/a, 2.0 implementation of client and provider for Flask.
      • OAuthLib – A generic and thorough implementation of the OAuth request-signing logic.
      • python-oauth2 – A fully tested, abstract interface to creating OAuth clients and servers.
      • python-social-auth – An easy-to-setup social authentication mechanism.
      • rauth – A Python library for OAuth 1.0/a, 2.0, and Ofly.
      • sanction – A dead simple OAuth2 client implementation.
    • Others
      • jose – JavaScript Object Signing and Encryption draft implementation.
      • PyJWT – Implementation of the JSON Web Token draft 01.
      • python-jws – Implementation of JSON Web Signatures draft 02.
      • python-jwt – Module for generating and verifying JSON Web Tokens.

    Build Tools

    Compile software from source code.

    • BitBake – A make-like build tool for embedded Linux.
    • buildout – A build system for creating, assembling and deploying applications from multiple parts.
    • PlatformIO – A console tool to build code with different development platforms.
    • PyBuilder – A continuous build tool written in pure Python.
    • SCons – A software construction tool.

    CMS

    Content Management Systems.

    • django-cms – An Open source enterprise CMS based on the Django.
    • djedi-cms – A lightweight but yet powerful Django CMS with plugins, inline editing and performance in mind.
    • FeinCMS – One of the most advanced Content Management Systems built on Django.
    • Kotti – A high-level, Pythonic web application framework built on Pyramid.
    • Mezzanine – A powerful, consistent, and flexible content management platform.
    • Opps – A Django-based CMS for magazines, newspapers websites and portals with high-traffic.
    • Plone – A CMS built on top of the open source application server Zope.
    • Quokka – Flexible, extensible, small CMS powered by Flask and MongoDB.
    • Wagtail – A Django content management system.
    • Widgy – Last CMS framework, based on Django.

    Caching

    Libraries for caching data.

    • Beaker – A library for caching and sessions for use with web applications and stand-alone Python scripts and applications.
    • DiskCache – SQLite and file backed cache backend with faster lookups than memcached and redis.
    • django-cache-machine – Automatic caching and invalidation for Django models.
    • django-cacheops – A slick ORM cache with automatic granular event-driven invalidation.
    • django-viewlet – Render template parts with extended cache control.
    • dogpile.cache – dogpile.cache is next generation replacement for Beaker made by same authors.
    • HermesCache – Python caching library with tag-based invalidation and dogpile effect prevention.
    • johnny-cache – A caching framework for django applications.
    • pylibmc – A Python wrapper around the libmemcached interface.

    ChatOps Tools

    Libraries for chatbot development.

    • Errbot – The easiest and most popular chatbot to implement ChatOps.

    Code Analysis and Linter

    Libraries and tools for analysing, parsing and manipulation codebases.

    • Code Analysis
      • coala – Language independent and easily extendable code analysis application.
      • code2flow – Turn your Python and JavaScript code into DOT flowcharts.
      • pycallgraph – A library that visualises the flow (call graph) of your Python application.
      • pysonar2 – A type inferencer and indexer for Python.
    • Linter
      • Flake8 – The modular source code checker: pep8, pyflakes and co.
      • pylama – Code audit tool for Python and JavaScript.
      • Pylint – A Fully customizable source code analyzer.

    Command-line Tools

    Libraries for building command-line application.

    • Command-line Application Development
      • asciimatics – Cross-platform, full-screen terminal package (i.e. mouse/keyboard input and coloured, positioned text output) complete with high-level API for complex animations and special effects.
      • cement – CLI Application Framework for Python.
      • click – A package for creating beautiful command line interfaces in a composable way.
      • cliff – A framework for creating command-line programs with multi-level commands.
      • clint – Python Command-line Application Tools.
      • colorama – Cross-platform colored terminal text.
      • docopt – Pythonic command line arguments parser.
      • Gooey – Turn command line programs into a full GUI application with one line
      • python-prompt-toolkit – A Library for building powerful interactive command lines.
    • Productivity Tools
      • aws-cli – A universal command-line interface for Amazon Web Services.
      • bashplotlib – Making basic plots in the terminal.
      • caniusepython3 – Determine what projects are blocking you from porting to Python 3.
      • cookiecutter – A command-line utility that creates projects from cookiecutters (project templates).
      • doitlive – A tool for live presentations in the terminal.
      • howdoi – Instant coding answers via the command line.
      • httpie – A command line HTTP client, a user-friendly cURL replacement.
      • mycli – A Terminal Client for MySQL with AutoCompletion and Syntax Highlighting.
      • PathPicker – Select files out of bash output.
      • percol – Adds flavor of interactive selection to the traditional pipe concept on UNIX.
      • pgcli – Postgres CLI with autocompletion and syntax highlighting.
      • SAWS – A Supercharged AWS CLI.
      • thefuck – Correcting your previous console command.
      • try – A dead simple CLI to try out python packages – It’s never been easier.

    Compatibility

    Libraries for migrating from Python 2 to 3.

    • Python-Future – The missing compatibility layer between Python 2 and Python 3.
    • Python-Modernize – Modernizes Python code for eventual Python 3 migration.
    • Six – Python 2 and 3 compatibility utilities.

    Computer Vision

    Libraries for computer vision.

    Concurrency and Parallelism

    Libraries for concurrent and parallel execution.

    • eventlet – Asynchronous framework with WSGI support.
    • gevent – A coroutine-based Python networking library that uses greenlet.
    • multiprocessing – (Python standard library) Process-based “threading” interface.
    • threading – (Python standard library) Higher-level threading interface.
    • Tomorrow – Magic decorator syntax for asynchronous code.
    • uvloop – Ultra fast implementation of asyncio event loop on top of libuv.

    Configuration

    Libraries for storing and parsing configuration options.

    • config – Hierarchical config from the author of logging.
    • ConfigObj – INI file parser with validation.
    • ConfigParser – (Python standard library) INI file parser.
    • profig – Config from multiple formats with value conversion.
    • python-decouple – Strict separation of settings from code.

    Cryptography

    • cryptography – A package designed to expose cryptographic primitives and recipes to Python developers.
    • hashids – Implementation of hashids in Python.
    • Paramiko – A Python (2.6+, 3.3+) implementation of the SSHv2 protocol, providing both client and server functionality.
    • Passlib – Secure password storage/hashing library, very high level.
    • PyCrypto – The Python Cryptography Toolkit.
    • PyNacl – Python binding to the Networking and Cryptography (NaCl) library.

    Data Analysis

    Libraries for data analyzing.

    • Blaze – NumPy and Pandas interface to Big Data.
    • Open Mining – Business Intelligence (BI) in Pandas interface.
    • Orange – Data mining, data visualization, analysis and machine learning through visual programming or scripts.
    • Pandas – A library providing high-performance, easy-to-use data structures and data analysis tools.

    Data Validation

    Libraries for validating data. Used for forms in many cases.

    • Cerberus – A lightweight and extensible data validation library.
    • colander – Validating and deserializing data obtained via XML, JSON, an HTML form post.
    • jsonschema – An implementation of JSON Schema for Python.
    • schema – A library for validating Python data structures.
    • Schematics – Data Structure Validation.
    • valideer – Lightweight extensible data validation and adaptation library.
    • voluptuous – A Python data validation library.

    Data Visualization

    Libraries for visualizing data. See: awesome-javascript.

    • Altair – Declarative statistical visualization library for Python.
    • Bokeh – Interactive Web Plotting for Python.
    • ggplot – Same API as ggplot2 for R.
    • Matplotlib – A Python 2D plotting library.
    • Pygal – A Python SVG Charts Creator.
    • PyGraphviz – Python interface to Graphviz.
    • PyQtGraph – Interactive and realtime 2D/3D/Image plotting and science/engineering widgets.
    • Seaborn – Statistical data visualization using Matplotlib.
    • VisPy – High-performance scientific visualization based on OpenGL.

    Database

    Databases implemented in Python.

    • pickleDB – A simple and lightweight key-value store for Python.
    • PipelineDB – The Streaming SQL Database.
    • TinyDB – A tiny, document-oriented database.
    • ZODB – A native object database for Python. A key-value and object graph database.

    Database Drivers

    Libraries for connecting and operating databases.

    • MySQL – awesome-mysql
      • mysql-python – The MySQL database connector for Python.
      • mysqlclient – mysql-python fork supporting Python 3.
      • oursql – A better MySQL connector with support for native prepared statements and BLOBs.
      • PyMySQL – Pure Python MySQL driver compatible to mysql-python.
    • PostgreSQL
      • psycopg2 – The most popular PostgreSQL adapter for Python.
      • queries – A wrapper of the psycopg2 library for interacting with PostgreSQL.
      • txpostgres – Twisted based asynchronous driver for PostgreSQL.
    • Other Relational Databases
      • apsw – Another Python SQLite wrapper.
      • dataset – Store Python dicts in a database – works with SQLite, MySQL, and PostgreSQL.
      • pymssql – A simple database interface to Microsoft SQL Server.
    • NoSQL Databases
      • cassandra-python-driver – Python driver for Cassandra.
      • HappyBase – A developer-friendly library for Apache HBase.
      • Plyvel – A fast and feature-rich Python interface to LevelDB.
      • py2neo – Python wrapper client for Neo4j’s restful interface.
      • pycassa – Python Thrift driver for Cassandra.
      • PyMongo – The official Python client for MongoDB.
      • redis-py – The Redis Python Client.
      • telephus – Twisted based client for Cassandra.
      • txRedis – Twisted based client for Redis.

    Date and Time

    Libraries for working with dates and times.

    • arrow – Better dates & times for Python.
    • Chronyk – A Python 3 library for parsing human-written times and dates.
    • dateutil – Extensions to the standard Python datetime module.
    • delorean – A library for clearing up the inconvenient truths that arise dealing with datetimes.
    • moment – A Python library for dealing with dates/times. Inspired by Moment.js.
    • Pendulum – Python datetimes made easy.
    • PyTime – A easy-use Python module which aims to operate date/time/datetime by string.
    • pytz – World timezone definitions, modern and historical. Brings the tz database into Python.
    • when.py – Providing user-friendly functions to help perform common date and time actions.

    Debugging Tools

    Libraries for debugging code.

    • Debugger
      • django-debug-toolbar – Display various debug information for Django.
      • django-devserver – A drop-in replacement for Django’s runserver.
      • flask-debugtoolbar – A port of the django-debug-toolbar to flask.
      • ipdb – IPython-enabled pdb.
      • pudb – A full-screen, console-based Python debugger.
      • pyringe – Debugger capable of attaching to and injecting code into Python processes.
      • wdb – An improbable web debugger through WebSockets.
      • winpdb – A Python Debugger with GUI, capable of remote debugging based on rpdb2.
    • Profiler
    • Others

    Deep Learning

    Frameworks for Neural Networks and Deep Learning. See: awesome-deep-learning.

    • Caffe – A fast open framework for deep learning..
    • Keras – A high-level neural networks library and capable of running on top of either TensorFlow or Theano.
    • Neupy – Running and testing different Artificial Neural Networks algorithms.
    • TensorFlow – The most popular Deep Learning framework created by Google.
    • Theano – A library for fast numerical computation.

    DevOps Tools

    Software and libraries for DevOps.

    • Ansible – A radically simple IT automation platform.
    • Cloud-Init – A multi-distribution package that handles early initialization of a cloud instance.
    • cuisine – Chef-like functionality for Fabric.
    • Docker Compose – Fast, isolated development environments using Docker.
    • Fabric – A simple, Pythonic tool for remote execution and deployment.
    • Fabtools – Tools for writing awesome Fabric files.
    • honcho – A Python clone of Foreman, for managing Procfile-based applications.
    • OpenStack – Open source software for building private and public clouds.
    • pexpect – Controlling interactive programs in a pseudo-terminal like GNU expect.
    • psutil – A cross-platform process and system utilities module.
    • SaltStack – Infrastructure automation and management system.
    • supervisor – Supervisor process control system for UNIX.

    Distribution

    Libraries to create packaged executables for release distribution.

    • dh-virtualenv – Build and distribute a virtualenv as a Debian package.
    • Nuitka – Compile scripts, modules, packages to an executable or extension module.
    • py2app – Freezes Python scripts (Mac OS X).
    • py2exe – Freezes Python scripts (Windows).
    • PyInstaller – Converts Python programs into stand-alone executables (cross-platform).
    • pynsist – A tool to build Windows installers, installers bundle Python itself.

    Documentation

    Libraries for generating project documentation.

    • Sphinx – Python Documentation generator.
    • MkDocs – Markdown friendly documentation generator.
    • pdoc – Epydoc replacement to auto generate API documentation for Python libraries.
    • Pycco – The literate-programming-style documentation generator.

    Downloader

    Libraries for downloading.

    • s3cmd – A command line tool for managing Amazon S3 and CloudFront.
    • s4cmd – Super S3 command line tool, good for higher performance.
    • you-get – A YouTube/Youku/Niconico video downloader written in Python 3.
    • youtube-dl – A small command-line program to download videos from YouTube.

    E-commerce

    Frameworks and libraries for e-commerce and payments.

    • alipay – Unofficial Alipay API for Python.
    • Cartridge – A shopping cart app built using the Mezzanine.
    • django-oscar – An open-source e-commerce framework for Django.
    • django-shop – A Django based shop system.
    • merchant – A Django app to accept payments from various payment processors.
    • money – Money class with optional CLDR-backed locale-aware formatting and an extensible currency exchange solution.
    • python-currencies – Display money format and its filthy currencies.
    • shoop – An open source E-Commerce platform based on Django.

    Editor Plugins and IDEs

    • Emacs
      • Elpy – Emacs Python Development Environment.
    • Sublime Text
      • Anaconda – Anaconda turns your Sublime Text 3 in a full featured Python development IDE.
      • SublimeJEDI – A Sublime Text plugin to the awesome auto-complete library Jedi.
    • Vim
      • Jedi-vim – Vim bindings for the Jedi auto-completion library for Python.
      • Python-mode – An all in one plugin for turning Vim into a Python IDE.
      • YouCompleteMe – Includes Jedi-based completion engine for Python.
    • Visual Studio
      • PTVS – Python Tools for Visual Studio.
    • Visual Studio Code
      • Python – An extension with rich support for the Python language, with features including linting, IntelliSense, formatting, refactoring, debugging, unit tests, and jupyter support.
      • Magic Python – Cutting edge Python syntax highlighter for Sublime Text, Atom, and Visual Studio Code. Used by GitHub to highlight your Python code!
    • IDE
      • LiClipse – Free polyglot IDE based on Eclipse. Uses PyDev for Python support.
      • PyCharm – Commercial Python IDE by JetBrains. Has free community edition available.
      • Spyder – Open Source Python IDE.

    Email

    Libraries for sending and parsing email.

    • envelopes – Mailing for human beings.
    • flanker – A email address and Mime parsing library.
    • imbox – Python IMAP for Humans.
    • inbox.py – Python SMTP Server for Humans.
    • lamson – Pythonic SMTP Application Server.
    • Marrow Mailer – High-performance extensible mail delivery framework.
    • modoboa – A mail hosting and management platform including a modern and simplified Web UI.
    • Nylas Sync Engine – Providing a RESTful API on top of a powerful email sync platform.
    • yagmail – Yet another Gmail/SMTP client.

    Environment Management

    Libraries for Python version and environment management.

    • p – Dead simple interactive Python version management.
    • pyenv – Simple Python version management.
    • venv – (Python standard library in Python 3.3+) Creating lightweight virtual environments.
    • virtualenv – A tool to create isolated Python environments.
    • virtualenvwrapper – A set of extensions to virtualenv.

    Files

    Libraries for file manipulation and MIME type detection.

    • imghdr – (Python standard library) Determine the type of an image.
    • mimetypes – (Python standard library) Map filenames to MIME types.
    • path.py – A module wrapper for os.path.
    • pathlib – (Python standard library in Python 3.4+) An cross-platform, object-oriented path library.
    • python-magic – A Python interface to the libmagic file type identification library.
    • Unipath – An object-oriented approach to file/directory operations.
    • watchdog – API and shell utilities to monitor file system events.

    Foreign Function Interface

    Libraries for providing foreign function interface.

    • cffi – Foreign Function Interface for Python calling C code.
    • ctypes – (Python standard library) Foreign Function Interface for Python calling C code.
    • PyCUDA – A Python wrapper for Nvidia’s CUDA API.
    • SWIG – Simplified Wrapper and Interface Generator.

    Forms

    Libraries for working with forms.

    • Deform – Python HTML form generation library influenced by the formish form generation library.
    • django-bootstrap3 – Bootstrap 3 integration with Django.
    • django-crispy-forms – A Django app which lets you create beautiful forms in a very elegant and DRY way.
    • django-remote-forms – A platform independent Django form serializer.
    • WTForms – A flexible forms validation and rendering library.

    Functional Programming

    Functional Programming with Python.

    • CyToolz – Cython implementation of Toolz: High performance functional utilities.
    • fn.py – Functional programming in Python: implementation of missing features to enjoy FP.
    • funcy – A fancy and practical functional tools.
    • Toolz – A collection of functional utilities for iterators, functions, and dictionaries.

    GUI

    Libraries for working with graphical user interface applications.

    • curses – Built-in wrapper for ncurses used to create terminal GUI applications.
    • enaml – Creating beautiful user-interfaces with Declaratic Syntax like QML.
    • Flexx – Flexx is a pure Python toolkit for creating GUI’s, that uses web technology for its rendering.
    • kivy – A library for creating NUI applications, running on Windows, Linux, Mac OS X, Android and iOS.
    • pyglet – A cross-platform windowing and multimedia library for Python.
    • PyGObject – Python Bindings for GLib/GObject/GIO/GTK+ (GTK+3)
    • PyQt – Python bindings for the Qt cross-platform application and UI framework, with support for both Qt v4 and Qt v5 frameworks.
    • PySide – Python bindings for the Qt cross-platform application and UI framework, supporting the Qt v4 framework.
    • Tkinter – Tkinter is Python’s de-facto standard GUI package.
    • Toga – A Python native, OS native GUI toolkit.
    • urwid – A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc.
    • wxPython – A blending of the wxWidgets C++ class library with the Python.

    Game Development

    Awesome game development libraries.

    • Cocos2d – cocos2d is a framework for building 2D games, demos, and other graphical/interactive applications. It is based on pyglet.
    • Panda3D – 3D game engine developed by Disney and maintained by Carnegie Mellon’s Entertainment Technology Center. Written in C++, completely wrapped in Python.
    • Pygame – Pygame is a set of Python modules designed for writing games.
    • PyOgre – Python bindings for the Ogre 3D render engine, can be used for games, simulations, anything 3D.
    • PyOpenGL – Python ctypes bindings for OpenGL and it’s related APIs.
    • PySDL2 – A ctypes based wrapper for the SDL2 library.
    • RenPy – A Visual Novel engine.

    Geolocation

    Libraries for geocoding addresses and working with latitudes and longitudes.

    • django-countries – A Django app that provides country choices for use with forms, flag icons static files, and a country field for models.
    • GeoDjango – A world-class geographic web framework.
    • GeoIP – Python API for MaxMind GeoIP Legacy Database.
    • geojson – Python bindings and utilities for GeoJSON.
    • geopy – Python Geocoding Toolbox.
    • pygeoip – Pure Python GeoIP API.

    HTML Manipulation

    Libraries for working with HTML and XML.

    • BeautifulSoup – Providing Pythonic idioms for iterating, searching, and modifying HTML or XML.
    • bleach – A whitelist-based HTML sanitization and text linkification library.
    • cssutils – A CSS library for Python.
    • html5lib – A standards-compliant library for parsing and serializing HTML documents and fragments.
    • lxml – A very fast, easy-to-use and versatile library for handling HTML and XML.
    • MarkupSafe – Implements a XML/HTML/XHTML Markup safe string for Python.
    • pyquery – A jQuery-like library for parsing HTML.
    • untangle – Converts XML documents to Python objects for easy access.
    • WeasyPrint – A visual rendering engine for HTML and CSS that can export to PDF.
    • xmldataset – Simple XML Parsing.
    • xmltodict – Working with XML feel like you are working with JSON.

    HTTP

    Libraries for working with HTTP.

    • grequests – requests + gevent for asynchronous HTTP requests.
    • httplib2 – Comprehensive HTTP client library.
    • requests – HTTP Requests for Humans™.
    • treq – Python requests like API built on top of Twisted’s HTTP client.
    • urllib3 – A HTTP library with thread-safe connection pooling, file post support, sanity friendly.

    Hardware

    Libraries for programming with hardware.

    • ino – Command line toolkit for working with Arduino.
    • Pingo – Pingo provides a uniform API to program devices like the Raspberry Pi, pcDuino, Intel Galileo, etc.
    • Pyro – Python Robotics.
    • PyUserInput – A module for cross-platform control of the mouse and keyboard.
    • scapy – A brilliant packet manipulation library.
    • wifi – A Python library and command line tool for working with WiFi on Linux.

    High Performance

    Libraries for making Python faster.

    • Cython – Optimizing Static Compiler for Python. Uses type mixins to compile Python into C or C++ modules resulting in large performance gains.
    • Numba – Python JIT complier to LLVM aimed at scientific Python.
    • PeachPy – x86-64 assembler embedded in Python. Can be used as inline assembler for Python or as a stand-alone assembler for Windows, Linux, OS X, Native Client and Go.
    • PyPy – An implementation of Python in Python. The interpreter uses black magic to make Python very fast without having to add in additional type information.
    • Pyston – A Python implementation built using LLVM and modern JIT techniques with the goal of achieving good performance.
    • Pyjion – A JIT for Python based upon CoreCLR
    • Stackless Python – An enhanced version of the Python.

    Imagery

    Libraries for manipulating images.

    • hmap – Image histogram remapping.
    • imgSeek – A project for searching a collection of images using visual similarity.
    • nude.py – Nudity detection.
    • pagan – Retro identicon (Avatar) generation based on input string and hash.
    • pillow – Pillow is the friendly PIL fork.
    • pyBarcode – Create barcodes in Python without needing PIL.
    • pygram – Instagram-like image filters.
    • python-qrcode – A pure Python QR Code generator.
    • Quads – Computer art based on quadtrees.
    • scikit-image – A Python library for (scientific) image processing.
    • thumbor – A smart imaging service. It enables on-demand crop, re-sizing and flipping of images.
    • wand – Python bindings for MagickWand, C API for ImageMagick.

    Interactive Interpreter

    Interactive Python interpreters (REPL).

    Internationalization

    Libraries for working with i18n.

    • Babel – An internationalization library for Python.
    • PyICU – A wrapper of International Components for Unicode C++ library (ICU).

    Job Scheduler

    Libraries for scheduling jobs.

    • APScheduler – A light but powerful in-process task scheduler that lets you schedule functions.
    • django-schedule – A calendaring app for Django.
    • doit – A task runner and build tool.
    • gunnery – Multipurpose task execution tool for distributed systems with web-based interface.
    • Joblib – A set of tools to provide lightweight pipelining in Python.
    • Plan – Writing crontab file in Python like a charm.
    • schedule – Python job scheduling for humans.
    • Spiff – A powerful workflow engine implemented in pure Python.
    • TaskFlow – A Python library that helps to make task execution easy, consistent and reliable.

    Logging

    Libraries for generating and working with logs.

    • Eliot – Logging for complex & distributed systems.
    • logbook – Logging replacement for Python.
    • logging – (Python standard library) Logging facility for Python.
    • Raven – The Python client for Sentry.
    • Sentry – A realtime logging and aggregation server.

    Machine Learning

    Libraries for Machine Learning. See: awesome-machine-learning.

    MapReduce

    Frameworks and libraries for MapReduce.

    • dpark – Python clone of Spark, a MapReduce alike framework in Python.
    • dumbo – Python module that allows one to easily write and run Hadoop programs.
    • luigi – A module that helps you build complex pipelines of batch jobs.
    • mrjob – Run MapReduce jobs on Hadoop or Amazon Web Services.
    • PySpark – The Spark Python API.
    • streamparse – Run Python code against real-time streams of data. Integrates with Apache Storm.

    Microsoft Windows

    Python programming on Microsoft Windows.

    • Python(x,y) – Scientific-applications-oriented Python Distribution based on Qt and Spyder.
    • pythonlibs – Unofficial Windows binaries for Python extension packages.
    • PythonNet – Python Integration with the .NET Common Language Runtime (CLR).
    • PyWin32 – Python Extensions for Windows.
    • WinPython – Portable development environment for Windows 7/8.

    Miscellaneous

    Useful libraries or tools that don’t fit in the categories above.

    • blinker – A fast Python in-process signal/event dispatching system.
    • itsdangerous – Various helpers to pass trusted data to untrusted environments.
    • pluginbase – A simple but flexible plugin system for Python.
    • Pychievements – A framework for creating and tracking achievements.
    • Tryton – A general purpose business framework.

    Natural Language Processing

    Libraries for working with human languages.

    • Jieba – Chinese text segmentation.
    • langid.py – Stand-alone language identification system.
    • NLTK – A leading platform for building Python programs to work with human language data.
    • Pattern – A web mining module for the Python.
    • SnowNLP – A library for processing Chinese text.
    • spaCy – A library for industrial-strength natural language processing in Python and Cython.
    • TextBlob – Providing a consistent API for diving into common NLP tasks.
    • TextGrocery – A simple, efficient short-text classification tool based on LibLinear and Jieba.

    Network Virtualization

    Tools and libraries for Virtual Networking and SDN (Software Defined Networking).

    • Mininet – A popular network emulator and API written in Python.
    • POX – An open source development platform for Python-based Software Defined Networking (SDN) control applications, such as OpenFlow SDN controllers.
    • Pyretic – A member of the Frenetic family of SDN programming languages that provides powerful abstractions over network switches or emulators.
    • SDX Platform – SDN based IXP implementation that leverages Mininet, POX and Pyretic.

    Networking

    Libraries for networking programming.

    • asyncio – (Python standard library) Asynchronous I/O, event loop, coroutines and tasks.
    • diesel – Greenlet-based event I/O Framework for Python.
    • pulsar – Event-driven concurrent framework for Python.
    • pyzmq – A Python wrapper for the ZeroMQ message library.
    • Twisted – An event-driven networking engine.
    • txZMQ – Twisted based wrapper for the ZeroMQ message library.

    News Feed

    Libraries for building user’s activities.

    ORM

    Libraries that implement Object-Relational Mapping or data mapping techniques.

    Package Management

    Libraries for package and dependency management.

    • pip – The Python package and dependency manager.
    • conda – Cross-platform, Python-agnostic binary package manager.
    • Curdling – Curdling is a command line tool for managing Python packages.
    • pip-tools – A set of tools to keep your pinned Python dependencies fresh.
    • wheel – The new standard of Python distribution and are intended to replace eggs.

    Package Repositories

    Local PyPI repository server and proxies.

    • warehouse – Next generation Python Package Repository (PyPI).
    • bandersnatch – PyPI mirroring tool provided by Python Packaging Authority (PyPA).
    • devpi – PyPI server and packaging/testing/release tool.
    • localshop – Local PyPI server (custom packages and auto-mirroring of pypi).

    Permissions

    Libraries that allow or deny users access to data or functionality.

    • Carteblanche – Module to align code with thoughts of users and designers. Also magically handles navigation and permissions.
    • django-guardian – Implementation of per object permissions for Django 1.2+
    • django-rules – A tiny but powerful app providing object-level permissions to Django, without requiring a database.

    Processes

    Libraries for starting and communicating with OS processes.

    • envoy – Python subprocess for Humans™.
    • sarge – Yet another wrapper for subprocess.
    • sh – A full-fledged subprocess replacement for Python.

    Queue

    Libraries for working with event and task queues.

    • celery – An asynchronous task queue/job queue based on distributed message passing.
    • huey – Little multi-threaded task queue.
    • mrq – Mr. Queue – A distributed worker task queue in Python using Redis & gevent.
    • rq – Simple job queues for Python.
    • simpleq – A simple, infinitely scalable, Amazon SQS based queue.

    RESTful API

    Libraries for developing RESTful APIs.

    • Django
    • Flask
      • eve – REST API framework powered by Flask, MongoDB and good intentions.
      • flask-api-utils – Taking care of API representation and authentication for Flask.
      • flask-api – Browsable Web APIs for Flask.
      • flask-restful – Quickly building REST APIs for Flask.
      • flask-restless – Generating RESTful APIs for database models defined with SQLAlchemy.
    • Pyramid
      • cornice – A RESTful framework for Pyramid.
    • Framework agnostic
      • falcon – A high-performance framework for building cloud APIs and web app backends.
      • hug – A Python3 framework for cleanly exposing APIs over HTTP and the Command Line with automatic documentation and validation.
      • restless – Framework agnostic REST framework based on lessons learned from Tastypie.
      • ripozo – Quickly creating REST/HATEOAS/Hypermedia APIs.
      • sandman – Automated REST APIs for existing database-driven systems.

    RPC Servers

    RPC-compatible servers.

    Science

    Libraries for scientific computing.

    • astropy – A community Python library for Astronomy.
    • bcbio-nextgen – Providing best-practice pipelines for fully automated high throughput sequencing analysis.
    • bccb – Collection of useful code related to biological analysis.
    • Biopython – Biopython is a set of freely available tools for biological computation.
    • cclib – A library for parsing and interpreting the results of computational chemistry packages.
    • NetworkX – A high-productivity software for complex networks.
    • NIPY – A collection of neuroimaging toolkits.
    • NumPy – A fundamental package for scientific computing with Python.
    • Open Babel – A chemical toolbox designed to speak the many languages of chemical data.
    • PyDy – Short for Python Dynamics, used to assist with workflow in the modeling of dynamic motion.
    • PyMC – Markov Chain Monte Carlo sampling toolkit.
    • RDKit – Cheminformatics and Machine Learning Software.
    • SciPy – A Python-based ecosystem of open-source software for mathematics, science, and engineering.
    • statsmodels – Statistical modeling and econometrics in Python.
    • SymPy – A Python library for symbolic mathematics.
    • Zipline – A Pythonic algorithmic trading library.

    Search

    Libraries and software for indexing and performing search queries on data.

    Serialization

    Libraries for serializing complex data types

    • marshmallow – marshmallow is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, to and from native Python datatypes.

    Serverless Frameworks

    Frameworks for developing serverless Python code.

    • apex – Build, deploy, and manage AWS Lambda functions with ease.
    • python-lambda – A toolkit for developing and deploying Python code in AWS Lambda.
    • Zappa – A tool for deploying WSGI applications on AWS Lambda and API Gateway.

    Specific Formats Processing

    Libraries for parsing and manipulating specific text formats.

    • General
      • tablib – A module for Tabular Datasets in XLS, CSV, JSON, YAML.
    • Office
      • Marmir – Takes Python data structures and turns them into spreadsheets.
      • openpyxl – A library for reading and writing Excel 2010 xlsx/xlsm/xltx/xltm files.
      • pyexcel – Providing one API for reading, manipulating and writing csv, ods, xls, xlsx and xlsm files.
      • python-docx – Reads, queries and modifies Microsoft Word 2007/2008 docx files.
      • relatorio – Templating OpenDocument files.
      • unoconv – Convert between any document format supported by LibreOffice/OpenOffice.
      • XlsxWriter – A Python module for creating Excel .xlsx files.
      • xlwings – A BSD-licensed library that makes it easy to call Python from Excel and vice versa.
      • xlwt / xlrd – Writing and reading data and formatting information from Excel files.
    • PDF
      • PDFMiner – A tool for extracting information from PDF documents.
      • PyPDF2 – A library capable of splitting, merging and transforming PDF pages.
      • ReportLab – Allowing Rapid creation of rich PDF documents.
    • Markdown
      • Mistune – Fastest and full featured pure Python parsers of Markdown.
      • Python-Markdown – A Python implementation of John Gruber’s Markdown.
    • YAML
      • PyYAML – YAML implementations for Python.
    • CSV
      • csvkit – Utilities for converting to and working with CSV.
    • Archive
      • unp – A command line tool that can unpack archives easily.

    Static Site Generator

    Static site generator is a software that takes some text + templates as input and produces HTML files on the output.

    • Cactus – Static site generator for designers.
    • Hyde – Jinja2-based static web site generator.
    • Lektor – An easy to use static CMS and blog engine.
    • Nikola – A static website and blog generator.
    • Pelican – Uses Markdown or ReST for content and Jinja 2 for themes. Supports DVCS, Disqus. AGPL.
    • Tinkerer – Tinkerer is a blogging engine/.static website generator powered by Sphinx.

    Tagging

    Libraries for tagging items.

    Template Engine

    Libraries and tools for templating and lexing.

    • Genshi – Python templating toolkit for generation of web-aware output.
    • Jinja2 – A modern and designer friendly templating language.
    • Mako – Hyperfast and lightweight templating for the Python platform.

    Testing

    Libraries for testing codebases and generating test data.

    • Testing Frameworks
      • hypothesis – Hypothesis is an advanced Quickcheck style property based testing library.
      • mamba – The definitive testing tool for Python. Born under the banner of BDD.
      • nose – A nicer unittest for Python.
      • nose2 – The successor to nose, based on unittest2.
      • pytest – A mature full-featured Python testing tool.
      • Robot Framework – A generic test automation framework.
      • unittest – (Python standard library) Unit testing framework.
    • Test Runners
      • green – A clean, colorful test runner.
      • tox – Auto builds and tests distributions in multiple Python versions
    • GUI / Web Testing
      • locust – Scalable user load testing tool written in Python.
      • PyAutoGUI – PyAutoGUI is a cross-platform GUI automation Python module for human beings.
      • Selenium – Python bindings for Selenium WebDriver.
      • sixpack – A language-agnostic A/B Testing framework.
      • splinter – Open source tool for testing web applications.
    • Mock
      • doublex – Powerful test doubles framework for Python.
      • freezegun – Travel through time by mocking the datetime module.
      • httmock – A mocking library for requests for Python 2.6+ and 3.2+.
      • httpretty – HTTP request mock tool for Python.
      • mock – (Python standard library) A mocking and patching library.
      • responses – A utility library for mocking out the requests Python library.
      • VCR.py – Record and replay HTTP interactions on your tests.
    • Object Factories
      • factory_boy – A test fixtures replacement for Python.
      • mixer – Another fixtures replacement. Supported Django, Flask, SQLAlchemy, Peewee and etc.
      • model_mommy – Creating random fixtures for testing in Django.
    • Code Coverage
      • coverage – Code coverage measurement.
    • Fake Data
      • church – is a Python library that help you generate fake data.
      • fake2db – Fake database generator.
      • faker – A Python package that generates fake data.
      • radar – Generate random datetime / time.
    • Error Handler
      • FuckIt.py – FuckIt.py uses state-of-the-art technology to make sure your Python code runs whether it has any right to or not.

    Text Processing

    Libraries for parsing and manipulating plain texts.

    • General
      • chardet – Python 2/3 compatible character encoding detector.
      • difflib – (Python standard library) Helpers for computing deltas.
      • ftfy – Makes Unicode text less broken and more consistent automagically.
      • fuzzywuzzy – Fuzzy String Matching.
      • Levenshtein – Fast computation of Levenshtein distance and string similarity.
      • pangu.py – Spacing texts for CJK and alphanumerics.
      • pyfiglet – An implementation of figlet written in Python.
      • shortuuid – A generator library for concise, unambiguous and URL-safe UUIDs.
      • unidecode – ASCII transliterations of Unicode text.
      • uniout – Print readable chars instead of the escaped string.
      • xpinyin – A library to translate Chinese hanzi (漢字) to pinyin (拼音).
    • Slugify
      • awesome-slugify – A Python slugify library that can preserve unicode.
      • python-slugify – A Python slugify library that translates unicode to ASCII.
      • unicode-slugify – A slugifier that generates unicode slugs with Django as a dependency.
    • Parser
      • phonenumbers – Parsing, formatting, storing and validating international phone numbers.
      • PLY – Implementation of lex and yacc parsing tools for Python
      • Pygments – A generic syntax highlighter.
      • pyparsing – A general purpose framework for generating parsers.
      • python-nameparser – Parsing human names into their individual components.
      • python-user-agents – Browser user agent parser.
      • sqlparse – A non-validating SQL parser.

    Third-party APIs

    Libraries for accessing third party services APIs. See: List of Python API Wrappers and Libraries.

    • apache-libcloud – One Python library for all clouds.
    • boto3 – Python interface to Amazon Web Services.
    • django-wordpress – WordPress models and views for Django.
    • facebook-sdk – Facebook Platform Python SDK.
    • facepy – Facepy makes it really easy to interact with Facebook’s Graph API
    • gmail – A Pythonic interface for Gmail.
    • google-api-python-client – Google APIs Client Library for Python.
    • gspread – Google Spreadsheets Python API.
    • twython – A Python wrapper for the Twitter API.

    URL Manipulation

    Libraries for parsing URLs.

    • furl – A small Python library that makes manipulating URLs simple.
    • purl – A simple, immutable URL class with a clean API for interrogation and manipulation.
    • pyshorteners – A pure Python URL shortening lib.
    • short_url – Python implementation for generating Tiny URL and bit.ly-like URLs.
    • webargs – A friendly library for parsing HTTP request arguments, with built-in support for popular web frameworks, including Flask, Django, Bottle, Tornado, and Pyramid.

    Video

    Libraries for manipulating video and GIFs.

    • moviepy – A module for script-based movie editing with many formats, including animated GIFs.
    • scikit-video – Video processing routines for SciPy.

    WSGI Servers

    WSGI-compatible web servers.

    • bjoern – Asynchronous, very fast and written in C.
    • fapws3 – Asynchronous (network side only), written in C.
    • gunicorn – Pre-forked, partly written in C.
    • meinheld – Asynchronous, partly written in C.
    • netius – Asynchronous, very fast.
    • paste – Multi-threaded, stable, tried and tested.
    • rocket – Multi-threaded.
    • uWSGI – A project aims at developing a full stack for building hosting services, written in C.
    • waitress – Multi-threaded, powers Pyramid.
    • Werkzeug – A WSGI utility library for Python that powers Flask and can easily be embedded into your own projects.

    Web Content Extracting

    Libraries for extracting web contents.

    • Haul – An Extensible Image Crawler.
    • html2text – Convert HTML to Markdown-formatted text.
    • lassie – Web Content Retrieval for Humans.
    • micawber – A small library for extracting rich content from URLs.
    • newspaper – News extraction, article extraction and content curation in Python.
    • opengraph – A Python module to parse the Open Graph Protocol
    • python-goose – HTML Content/Article Extractor.
    • python-readability – Fast Python port of arc90’s readability tool.
    • sanitize – Bringing sanity to world of messed-up data.
    • sumy – A module for automatic summarization of text documents and HTML pages.
    • textract – Extract text from any document, Word, PowerPoint, PDFs, etc.

    Web Crawling

    Libraries for scraping websites.

    • cola – A distributed crawling framework.
    • Demiurge – PyQuery-based scraping micro-framework.
    • feedparser – Universal feed parser.
    • Grab – Site scraping framework.
    • MechanicalSoup – A Python library for automating interaction with websites.
    • portia – Visual scraping for Scrapy.
    • pyspider – A powerful spider system.
    • RoboBrowser – A simple, Pythonic library for browsing the web without a standalone web browser.
    • Scrapy – A fast high-level screen scraping and web crawling framework.

    Web Frameworks

    Full stack web frameworks.

    • Bottle – A fast, simple and lightweight WSGI micro web-framework.
    • CherryPy – A minimalist Python web framework, HTTP/1.1-compliant and WSGI thread-pooled.
    • Django – The most popular web framework in Python.
    • Flask – A microframework for Python.
    • Pyramid – A small, fast, down-to-earth, open source Python web framework.
    • Tornado – A Web framework and asynchronous networking library.
    • TurboGears – A microframework that can scale up to a full stack solution.

    WebSocket

    Libraries for working with WebSocket.

    Services

    Online tools and APIs to simplify development.

    Continuous Integration

    See: awesome-CIandCD.

    • CircleCI – A CI service that can run very fast parallel testing. (GitHub only)
    • Travis CI – A popular CI service for your open source and private projects. (GitHub only)
    • Vexor CI – A continuous integration tool for private apps with pay-per-minute billing model.
    • Wercker – A Docker-based platform for building and deploying applications and microservices.

    Code Quality

    • Codacy – Automated Code Review to ship better code, faster. Free for Open Source.
    • Codecov – Code coverage dashboard.
    • Landscape – Hosted continuous Python code metrics.
    • QuantifiedCode – A data-driven, automated, continuous code review tool.

    Resources

    Where to discover new Python libraries.

    Podcasts

    Twitter

    Websites

    Weekly

    Other Awesome Lists

    List of lists.

    Reference: https://github.com/vinta/awesome-python

    No system is truly secure!

You must be logged in to reply to this topic.