Miscellaneous Python Resources

Dr. Phillip M. Feldman

1. console_input.py

This module defines several general-purpose functions:


console_input.py


2. CustomImporter.py

2.1 Overview

This module defines two main functions—Cimport and Cfrom—that are similar to but slightly more flexible than Python's built-in import and from commands, respectively, as well as several ancillary functions. The behavior of Cimport and Cfrom differs from that of their built-in counterparts as follows:
  1. The calling program can specify a list of folders to be searched for the module. If such a list is provided, these folders are searched first, and any folders specified via import_path are searched subsequently.
  2. The module name may be fully-qualified, in which case no search is performed.

The contents of import_path can be set by calling the setpath function; by default, the contents of import_path are the same as those of sys.path (the Python path).

2.2 Instructions and Examples

In example #s 1-4, the new style of import provides no advantage over the old one, and the old style is actually preferrable since one less line of code is required.

Example #1

Old:
   import xyz
New:
   from CustomImporter import Cimport
   xyz= Cimport('xyz')

Example #2

Old:
   import A as B
New:
   from CustomImporter import Cimport
   B= Cimport('A')

Example #3

Old:
   from xyz import abc
New:
   from CustomImporter import Cfrom
   abc= Cfrom('xyz', 'abc')

Example #4

Old:
   from A import B as C
New:
   from CustomImporter import Cfrom
   C= Cfrom('A', 'B')

Example #5

   from CustomImporter import Cimport
   C= Cimport('A', in_folders=['C:/Audio', 'C:/Video'], as_name='B')
In this last example, the call to Cimport causes it to search for a module called A in the two specified folders, and if it is not found there, in sys.path. If the module is found, it will be loaded and stored in sys.modules under the key 'B', with a reference being stored in C.


CustomImporter.py

3. prefixes1.py and prefixes2.py

These scripts process a spelling dictionary to identify commonly-occurring prefixes. For our immediate purposes, their value is in demonstrating many of the fundamentals of Python, including file input, looping, string operations, and operations involving lists and sets. Note that Python's for loop operates on elements of a list, much like Perl's foreach loop.

prefixes1.py and prefixes2.py

4. primes.py

This Python module contains functions for performing various calculations involving prime numbers, including primality testing, prime factorization, and the like. From the standpoint of learning Python, it demonstrates several things:

primes.py

5. Type.py

Python permits one to create hierarchical data structures with arbitrary nesting. (Data structures may even be recursive, in which case the depth of nesting is infinite). Although hierarchical data structures are extremely powerful, they can also be hard to understand. The module Type.py defines a function called Type that recursively explores an object to display information about data types at various levels in the hierarchy. This function can be used in any of several ways:

The following example demonstrates the use of Type. I define two data structures and apply the function to each. The first data structure is a list containing two integers and one float value. The second, more complex data structure is a dictionary whose values are a list and a nested dictionary.

In [1]: from Type import *
In [2]: x= [1, 2, 3.3]
In [3]: Type(x)
Out[3]: 'list[float,int]'

In [4]: x= {'A':[1,2,3], 'B':{'bb':2.2, 'cc':3.3}}
In [5]: Type(x)
Out[5]: 'dict[dict[float],list[int]]'
Type.py

6. fit_cubic.py

This script demonstrates how to solve a small system of nonlinear equations numerically using Python and SciPy, as well as how to perform basic operations on polynomials. The script does the following:

(1) It uses one of SciPy's root solvers to find the cubic polynomial p(x) such that the polynomial and its derivative q(x) have specified values for two values of x.

(2) It plots the resulting polynomial. Inputs are hardwired into the code; one can modify them by editing Section 2.

fit_cubic.py

7. quantile_regression_demo.py

7.1 Overview

Most of us are familiar with the charts that pediatricians use that show percentiles of weight and height as a function of age; generating such a chart from a small sample of data requires quantile regression or similar methods. (When working with a large enough sample of data, one can bin the data, i.e., divide the x-axis into intervals and calculate percentiles independently for each interval. But, this approach uses the data inefficiently and is unworkable when sample sizes are small).

Quantiles and percentiles are the same except for a factor of 100, e.g., the 30th percentile is the 0.3 quantile.

This Python script demonstrates that one can perform quantile regression using only Python, NumPy, and SciPy. The only other dependency is on matplotlib, which is used to plot the data and the quantile estimates.

7.2 Details

In detail, the script does the following:

(1) Model parameters are assigned. (Currently, these are hardwired into the code).

(2) The program generates an artificial bivariate sample of data (x, y) as follows:

- x is generated by drawing from a distribution that is uniform on [x_min, x_max], where x_min and x_max are currently 0 and 1, respectively.

- y is then generated according to a normal distribution having mean -0.5 + x and standard deviation 1.0 + 0.5 * x.

(All of this can be changed, e.g., one could choose to make the mean of y quadratic in x).

(3) The code defines an objective function based on the tilted absolute value function (see references for motivation).

(4) The SciPy optimization package is then used to optimize (minimize) the objective function.

(5) Using the matplotlib module, the code plots a scatter diagram of the data with an overlay of percentile lines.

Note: Artificial data samples are an excellent way to verify that a statistical algorithm works under ideal conditions, and to explore robustness, i.e., algorithm behavior when standard assumptions such as Gaussianity and independence are (slightly) violated.

Additional notes can be found in the source code.

quantile_regression_demo.py

8. Designing Applications to Work Across Multiple Platforms

8.1 Introduction

Many Python applications work across platforms such as 64-bit Windows, 32-bit Windows, and 64-bit Linux without any special effort being taken to achieve platform independence. There are situations, however, where achieving compatibility across multiple platforms requires care. Here are a few considerations to keep in mind:

8.2 OS.py

My OS module defines a single function having the same name as the module. OS() returns a length-3 list of strings identifying the operating system environment:

  1. the OS family name as returned by platform.system, e.g., 'Windows' or 'Linux'.

  2. the OS release (major version identifier) as returned by platform.release, subject to the following possible modification: If a matching key appears in the release_dict dictionary (see the code), the version name will be replaced by the dictionary value. This dictionary is easy to customize.

  3. the OS word length in bits--either '32' or '64'.

OS.py

9. Fourier_series.py

This program demonstrates the use of the matplotlib graphical toolkit in conjunction with the Qt Graphical User Interface (GUI) API. matplotlib provides an extensive set of functions for generating various types of 2-D and 3-D graphics, but does not support user interaction. When writing a Python program that produces graphical output and must also handle user interaction (via keyboard and/or mouse), there are at least two basic approaches:

  1. Use matplotlib together with a GUI toolkit such as Qt or Wx. [I used Wx at one time, but have been migrating to Qt, which is more powerful and is rapidly supplanting Wx.]

  2. Use Chaco. "Chaco is a Python plotting application toolkit that facilitates writing plotting applications at all levels of complexity, from simple scripts with hard-coded data to large plotting programs with complex data interrelationships and a multitude of interactive tools. While Chaco generates attractive static plots for publication and presentation, it also works well for interactive data visualization and exploration."

For simple user interactions, (1) is the preferred approach.

This program generates successive Fourier series approximations to a square wave, triangle wave, sawtooth wave, full-wave rectified sine wave, or half-wave rectified sine wave. Sample output appears below.


The program recognizes the following keys:

Fourier_series.py

Last update: 12 June, 2015