# -*- coding: utf-8 -*-
# cxfreeze.py
"""
Overview
========
Creates a standalone program package for a particular platform to be run by
restricted users without installing any additional packages.
This script is executable and has to be run on the platform for which a
package shall be created. Please follow the instructions below for each
particular platform.
Common Package Dependencies:
- `Python 2.7 <https://www.python.org/download/releases/2.7/>`_
- `Qt 4.8 <http://qt-project.org/doc/qt-4.8/qt4-8-intro.html>`_ + `PySide <http://qt-project.org/wiki/Category:LanguageBindings::PySide::Downloads>`_
- `NumPy and SciPy <http://www.scipy.org/scipylib/download.html>`_
In order to work around freeze failures with newer versions it is
recommended to stick with Numpy 1.7 and SciPy 1.12 which was tested
successfully.
- `matplotlib <http://matplotlib.org/downloads.html>`_
In addition to the dependencies of the MCSAS package listed above the
`cx_Freeze package <http://cx-freeze.readthedocs.org/en/latest/>`_
is used for freezing the python source code structure into a standalone
package.
Working with Source Code Repositories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to download the latest source code repositories of packages such as
MCSAS or cx_Freeze a client to `Git`_ and `Mercurial`_ is required. There are
several available, for both Mac OS X and Windows the `SourceTree`_ program
is recommended.
.. _Git: https://en.wikipedia.org/wiki/Git_%28software%29
.. _Mercurial: https://en.wikipedia.org/wiki/Mercurial
.. _SourceTree: http://www.sourcetreeapp.com
Windows
-------
A self-contained archive consisting of ``MCSAS.exe`` and all necessary
libraries and files is created by the following command executed in the
MCSAS folder::
> python cxfreeze.py build_exe
Requirements
^^^^^^^^^^^^
On a fresh installation of Windows 7 the following packages are required:
- `Python 2.7.9 <https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi>`_
- `PySide 1.2.1 <https://download.qt.io/official_releases/pyside/PySide-1.2.1.win32-py2.7.exe>`_
- `NumPy 1.7.1 <http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/numpy-1.7.1-win32-superpack-python2.7.exe>`_
- `SciPy 0.12.0 <http://sourceforge.net/projects/scipy/files/scipy/0.12.0/scipy-0.12.0-win32-superpack-python2.7.exe>`_
- `matplotlib 1.4.2 <https://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.4.2/windows/matplotlib-1.4.2.win32-py2.7.exe>`_ and its requirements:
- `Six 1.9.0 <https://pypi.python.org/packages/3.3/s/six/six-1.9.0-py2.py3-none-any.whl>`_
Install it on the command line by::
pip install six-1.9.0-py2.py3-none-any.whl
- `dateutil 2.4.0 <https://pypi.python.org/packages/py2.py3/p/python-dateutil/python_dateutil-2.4.0-py2.py3-none-any.whl>`_
- `pyparsing 2.0.3 <http://sourceforge.net/projects/pyparsing/files/pyparsing/pyparsing-2.0.3/pyparsing-2.0.3.win32-py2.7.exe>`_
- `cx_Freeze 4.3.4 <https://pypi.python.org/packages/2.7/c/cx_Freeze/cx_Freeze-4.3.4.win32-py2.7.exe>`_
- `pywin32 219 <http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/pywin32-219.win32-py2.7.exe>`_
- h5py HDF5 support, install one of the precompiled Windows packages, such as
`h5py-2.4.0.win32-py2.7.exe <https://pypi.python.org/packages/df/6d/b9463b64fa8ff6bd71ae7c5a5b3591dbc1c8536af0926f0bfb712a4356fe/h5py-2.4.0.win32-py2.7.exe#md5=032d7d10a9938aefbcc278e2fe5e2864>`_
Mac OS X
--------
After installing the required packages below a disk image file (.dmg)
consisting of the application bundle is created by::
$ /usr/local/bin/python2 cxfreeze.py bdist_dmg
Alternatively, for testing purposes the bundle can be created without
packaging into a disk image by::
$ /usr/local/bin/python2 cxfreeze.py bdist_mac
Requirements
^^^^^^^^^^^^
On a fresh installation of OS X 10.8 the following packages are required:
- Xcode command line tools: for build essentials such as a compiler
( `xcode461_cltools_10_86938245a.dmg <https://developer.apple.com/downloads/download.action?path=Developer_Tools/command_line_tools_os_x_mountain_lion_for_xcode__march_2013/xcode461_cltools_10_86938245a.dmg>`_ )
- `Python 2.7.9 <https://www.python.org/ftp/python/2.7.9/python-2.7.9-macosx10.6.pkg>`_
- `Qt 4.8.6 <http://download.qt.io/official_releases/qt/4.8/4.8.6/qt-opensource-mac-4.8.6-1.dmg>`_
- `PySide 1.2.1 / Qt 4.8 <http://pyside.markus-ullmann.de/pyside-1.2.1-qt4.8.5-py27apple-developer-signed.pkg>`_
- `NumPy 1.7.1 <http://downloads.sourceforge.net/project/numpy/NumPy/1.7.1/numpy-1.7.1-py2.7-python.org-macosx10.6.dmg>`_
- `SciPy 0.12.0 <http://sourceforge.net/projects/scipy/files/scipy/0.12.0/scipy-0.12.0-py2.7-python.org-macosx10.6.dmg>`_
- `matplotlib 1.4.2 <https://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.4.2/mac/matplotlib-1.4.2-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl>`_
Install it on the command line by::
$ /usr/local/bin/pip install matplotlib-1.4.2-*.whl
- h5py HDF5 support, install HDF5 from source first::
$ cd hdf5-src
$ ./configure --prefix=/usr/local
$ make && sudo make install
$
$ pip2 install h5py
- `a modified cx_Freeze 4.3.4 <https://bitbucket.org/ibressler/cx_freeze>`_
with local modifications for successful app freezing on OS X
Download the source and install it on the command line by::
$ hg clone https://bitbucket.org/ibressler/cx_freeze
$ cd cx_freeze
$ hg co 4.x
$ /usr/local/bin/python2 setup.py install
Ubuntu/Linux
------------
Similar to the procedure on Windows a self-contained archive containing all
necessary libraries and files is created by::
$ python cxfreeze.py build_exe
Requirements
^^^^^^^^^^^^
On a fresh installation of Ubuntu Linux 14.04 LTS the following packages
are required:
- ``apt-get install git build-essential python-setuptools python-dev liblapack-dev libfreetype6-dev tk-dev``
- PySide 1.2.4
- NumPy 1.7.2
- SciPy 0.12.1
- matplotlib 1.4.2
- cx_Freeze 4.3.4
- future 0.16.1
- h5py 2.2.1
Internals
=========
"""
import sys
import logging
logging.basicConfig(level = logging.INFO)
import gui.version
OLDVERSIONNUM = gui.version.version.number()
if len(sys.argv) > 2 and __name__ == "__main__":
alternateVersion = "-".join(sys.argv[2:])
del sys.argv[2:]
logging.info("Using an alternate version: '{0}'".format(alternateVersion))
gui.version.version.updateFile(gui.version, alternateVersion)
reload(gui.version)
import re
import os
import subprocess
import hashlib
import platform
import tempfile
import glob
from cx_Freeze import setup, Executable
from gui.version import version
from utils import isWindows, isLinux, isMac, testfor, mcopen
from utils.findmodels import FindModels
[docs]def sanitizeVersionNumber(number):
"""Removes non-digits to be compatible with pywin32"""
if not isWindows():
return number
match = re.match(r'(\d+)[^\d](\d+)[^\d](\d+)', number)
if match is None:
return "0"
number = ".".join(match.groups())
try:
bits = [int(i) for i in number.split(".")]
assert(len(bits))
except (AssertionError, IndexError, TypeError, ValueError):
number = "0" # do not set version number in win32 binary
return number
[docs]class Archiver(object):
_name = None # to be defined by sub classes
_path = None
def __init__(self):
self._path = self._getPath()
testfor(self._path is not None and os.path.isfile(self._path),
OSError,
"{name}: '{path}' not found!".format(
name = self._name,
path = self._path))
@property
def execName(self):
return os.path.basename(self._path)
[docs] def getLogFilename(self):
return os.path.splitext(self.execName)[0] + ".log" # 7z.log
[docs] def archive(self, targetPath):
"""Creates an archive from the given absolute target directory path.
The archive file will have the base name of the last directory of the
given path.
"""
raise NotImplementedError
[docs]class Archiver7z(Archiver):
_name = "7-Zip"
_types = ("7z", "zip")
_ext = None
_type = None
def __init__(self, filetype = "7z"):
super(Archiver7z, self).__init__()
if filetype not in self._types:
filetype = self._types[0]
self._type = filetype
self._ext = filetype
@staticmethod
def _getPath():
"""Retrieves 7-Zip install path and binary from Windows Registry.
This way, we do not rely on PATH containing 7-Zip.
"""
path = None
if isWindows():
import win32api, win32con
for key in (win32con.HKEY_LOCAL_MACHINE, win32con.HKEY_CURRENT_USER):
try:
key = win32api.RegConnectRegistry(None, key)
key = win32api.RegOpenKeyEx(key, r"SOFTWARE\7-Zip")
path, dtype = win32api.RegQueryValueEx(key, "Path")
path = os.path.abspath(os.path.join(path, "7z.exe"))
except:
pass
else:
p = subprocess.Popen(["which", "7z"],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
out, err = p.communicate()
path = out.strip()
return path
[docs] def archive(self, targetPath):
targetPath = os.path.abspath(targetPath)
if not os.path.isdir(targetPath):
return None
fnPackage = targetPath + "." + self._ext
fnLog = os.path.abspath(self.getLogFilename())
with mcopen(fnLog, 'w') as fd:
retcode = subprocess.call(
[self._path, "a", "-t" + self._type, "-mx=9",
os.path.relpath(fnPackage),
os.path.relpath(targetPath)],
stdout = fd,
stderr = fd)
if not os.path.exists(fnPackage):
fnPackage = None
return fnPackage
[docs]class ArchiverZip(Archiver):
_name = "Zip"
@staticmethod
def _getPath():
path = None
if isWindows():
pass # none yet
else:
p = subprocess.Popen(["which", "zip"],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
out, err = p.communicate()
path = out.strip()
return path
[docs] def archive(self, targetPath):
"""Expects an absolute target directory path"""
targetPath = os.path.abspath(targetPath)
if not os.path.isdir(targetPath):
return None
fnPackage = targetPath + ".zip"
fnLog = os.path.abspath(self.getLogFilename())
with mcopen(fnLog, 'w') as fd:
retcode = subprocess.call([self._path, "-r9",
os.path.relpath(fnPackage),
os.path.relpath(targetPath)],
stdout = fd,
stderr = fd)
if not os.path.exists(fnPackage):
fnPackage = None
return fnPackage
[docs]def includeModels(includeFilesLst):
modelsDir = FindModels.rootName
for path, relFile in FindModels.candidateFiles():
src = os.path.join(path, relFile)
dst = os.path.join(modelsDir, relFile)
includeFilesLst.append((src, dst))
if __name__ == "__main__":
# using zip by default, its preinstalled everywhere
archiver = None
# TODO: perhaps switch to pythons builtin zip?
if isWindows():
archiver = Archiver7z(filetype = "zip")
else:
archiver = ArchiverZip()
PACKAGENAME = "{name} {ver}".format(
name = version.name(),
ver = version.number())
# target (temp) dir for mcsas package
sysname = platform.system().title()
if "darwin" in sysname.lower():
sysname = "MacOS"
TARGETDIR = "{pckg} for {sysname}".format(
pckg = PACKAGENAME, sysname = sysname)
if isLinux():
bitness, binfmt = platform.architecture()
TARGETDIR += " " + bitness
BASE = None
EXEC_SUFFIX = ""
if sys.platform in "win32":
BASE = "Win32GUI"
EXEC_SUFFIX = ".exe"
TARGETNAME = version.name() + EXEC_SUFFIX
# (source, target) pairs
# without a target the file is placed in the top level directory of the package
INCLUDEFILES = [
"mcsas/mcsasparameters.json",
("resources/background_files.svg", "resources/background_files.svg"),
("resources/background_ranges.svg", "resources/background_ranges.svg"),
("resources/icon/mcsas.ico", "resources/icon/mcsas.ico"),
"dejavuserif.ttf", "dejavumono.ttf",
]
# python3 compatibility fix
import lib2to3
lib23_path = os.path.dirname(lib2to3.__file__)
INCLUDEFILES.append(lib23_path)
# testing
includeModels(INCLUDEFILES)
if isLinux():
INCLUDEFILES += [
"/usr/lib/liblapack.so.3",
"/usr/lib/libblas.so.3",
]
libdir = "/usr/lib/x86_64-linux-gnu"
if not os.path.isdir(libdir):
libdir = "/usr/lib/i386-linux-gnu"
for lib in ("libgfortran.so.3", "libquadmath.so.0",
"libpyside-python2.7.so.1.2", "libshiboken-python2.7.so.1.2",
"libQtGui.so.4", "libQtCore.so.4", "libQtSvg.so.4", "libQtXml.so.4",
"libQtNetwork.so.4", "libQtDBus.so.4",
"libaudio.so.2", "libhdf5.so.7", "libhdf5_hl.so.7",
"libpng12.so.0", "libtk8.6.so", "libXss.so.1"):
filepath = os.path.join(libdir, lib)
if not os.path.exists(filepath):
logging.warning("include not found: '{}'".format(filepath))
else:
INCLUDEFILES.append(filepath)
if isWindows():
INCLUDEFILES += [
"Microsoft.VC90.CRT",
]
if isMac():
INCLUDEFILES += [
("resources/icon/mcsas.icns", "resources/icon/mcsas.icns"),
"/usr/lib/system/libdnsinfo.dylib"
]
BUILDOPTIONS = dict(
compressed = False,
include_files = INCLUDEFILES,
packages = [],
excludes = ["lib2to3", # python3 compatibility fix
"models.sphere"], # source file added for dyn. loading
includes = ["PySide", "PySide.QtCore", "PySide.QtGui",
"PySide.QtSvg", "PySide.QtXml",
"multiprocessing",
"scipy.sparse.csgraph._validation",
"scipy.special._ufuncs_cxx",
"scipy.sparse.linalg.dsolve.umfpack", "scipy.integrate.vode",
"scipy.integrate.lsoda", "matplotlib",
"matplotlib.backends.backend_qt4agg",
# savefig dependencies?
"matplotlib.backends.backend_tkagg", "Tkinter", "FileDialog",
"h5py",
"UserList", "UserString",
],
# Icons for Windows using this approach:
# https://stackoverflow.com/a/10819673
icon = "resources/icon/mcsas.ico",
path = [os.getcwd()] + sys.path,
build_exe = TARGETDIR,
silent = False,
copy_dependent_files = True,
)
# OSX bundle building
MACOPTIONS = dict(
bundle_name = TARGETDIR,
# creating icns for OSX: https://stackoverflow.com/a/20703594
iconfile = "resources/icon/mcsas.icns"
)
DMGOPTIONS = dict(volume_label = TARGETDIR,
applications_shortcut = True)
if isMac():
BUILDOPTIONS.pop("build_exe") # bdist_mac expects plain 'build' directory
BUILDOPTIONS["includes"] = [ # order in which they were requested
"PySide", "PySide.QtCore", "PySide.QtGui", "PySide.QtSvg", "PySide.QtXml",
"scipy.sparse.csgraph._validation",
"scipy.sparse.linalg.dsolve.umfpack",
"matplotlib.backends.backend_qt4agg",
"scipy.integrate.vode",
"scipy.integrate.lsoda",
"h5py", "UserList", "UserString",
]
BUILDOPTIONS.pop("icon")
# tcl/tk is installed by default
BUILDOPTIONS["bin_excludes"] = ["Tcl", "Tk", "libgcc_s.1.dylib"]
BUILDOPTIONS["excludes"] += ["Tkinter"]
os.environ["DYLD_FRAMEWORK_PATH"] = ":".join((
"/Library/Frameworks", "/System/Library/Frameworks"))
os.environ["DYLD_LIBRARY_PATH"] = ":".join((
"/usr/lib", "/usr/local/lib"))
setup(
name = version.name(),
version = sanitizeVersionNumber(version.number()),
description = version.name(),
long_description = ("GUI for Monte-Carlo size distribution analysis"),
license = "Creative Commons CC-BY-SA",
author = u"Brian R. Pauw",
author_email = "brian@stack.nl",
contact = u"Brian R. Pauw",
contact_email = "brian@stack.nl",
maintainer = u"Ingo Breßler",
maintainer_email = "ingo.bressler@bam.de",
# additional metadata for modified version of cx_Freeze
# displayed in copyright info
download_url = "2015, http://www.mcsas.net",
# company
classifiers = u"NIMS\r\n"
u"National Institute for Materials Science, \r\n\r\n"
u"1-2-1 Sengen, 305-0047, "
u"305-0047, Tsukuba, Japan",
options = dict(build_exe = BUILDOPTIONS,
bdist_mac = MACOPTIONS,
bdist_dmg = DMGOPTIONS),
executables = [Executable("main.py", base = BASE,
targetName = TARGETNAME)])
# calc a checksum of the package
def hashFile(filename, hasher, blocksize = 65536):
os.path.abspath(filename)
with mcopen(filename, 'rb') as fd:
buf = fd.read(blocksize)
while len(buf) > 0:
hasher.update(buf)
buf = fd.read(blocksize)
return hasher.hexdigest(), os.path.basename(filename)
# package the freezed program into an archive file
PACKAGEFN = archiver.archive(TARGETDIR)
if PACKAGEFN is None and os.path.isdir("build"):
# find package created by bdist_mac or bdist_dmg commands
files = glob.glob("build/*.dmg")
if not len(files):
files = glob.glob("build/*.app")
if len(files):
PACKAGEFN = files[0]
# rename package possibly
fn, ext = os.path.splitext(PACKAGEFN)
if os.path.exists(PACKAGEFN) and fn != TARGETDIR:
try:
newfn = "".join((TARGETDIR, ext))
os.rename(PACKAGEFN, newfn)
PACKAGEFN = newfn
except OSError as e:
logging.exception("rename '{src}' -> '{dst}':"
.format(src = PACKAGEFN, dst = newfn))
pass
logging.info("Created package '{0}'.".format(PACKAGEFN))
if PACKAGEFN is not None and os.path.isfile(PACKAGEFN):
hashValue = hashFile(PACKAGEFN, hashlib.sha256())
# write the checksum to file
with mcopen(os.path.abspath(PACKAGEFN + ".sha256"), 'w') as fd:
fd.write(" *".join(hashValue))
# always restore initially modified version
if version.number() != OLDVERSIONNUM:
version.updateFile(gui.version, OLDVERSIONNUM)
logging.info("done.")
# vim: set ts=4 sw=4 sts=4 tw=0: