.. _installation_deployment:

Deployment Environment
======================

Installing a Global Virtual Environment
---------------------------------------

It is best practice to install `PubliForge` into a "virtual" Python environment
in order to obtain isolation from any "system" packages you've got installed in
your Python version. This can be done by using the `virtualenv
<http://pypi.python.org/pypi/virtualenv>`_ package.  Using a virtualenv will
also prevent `PubliForge` from globally installing versions of packages that
are not compatible with your system Python.

This documentation assumes that Python and virtualenv are already installed and
working in your system.

Install a global virtual environment for all your instances of `PubliForge`. To
do so, invoke the following:

.. code-block:: bash

   $ sudo virtualenv /usr/local/virtualenv
   $ sudo chown $USER -R /usr/local/virtualenv
   $ source /usr/local/virtualenv/bin/activate
   (virtualenv)$ pip install -U pip setuptools

Adding a User
-------------

Create a user for your instance so you can control use of resources
(Cf. ``ulimit``, ``quota``...). In this documentation, we suppose the name of
this user is ``pfinstance``.

.. code-block:: bash

    $ sudo adduser pfinstance

Make sure that on your system an UTF-8 locale is generated. To do so, type:

.. code-block:: bash

    $ sudo dpkg-reconfigure locales


.. _installation_deployment_packages:

Installing Required Packages
----------------------------

Before installing `PubliForge`, you must have on your system the following
packages:

.. code-block:: bash

    $ sudo aptitude install python-dev libxml2-dev libxslt1-dev librsync-dev libpq-dev libaprutil1-dev libsvn-dev

If you plan to use a MySQL Database:

.. code-block:: bash

    $ sudo aptitude install libmysqlclient-dev

For `LePrisme` module, you also need the following packages:

.. code-block:: bash

    $ sudo aptitude install yui-compressor
    $ sudo aptitude install imagemagick optipng advancecomp jpegoptim
    $ sudo aptitude install ffmpeg

If you plan to make ePub, you will need a Java Runtime Environment to run
`ePubCheck <http://code.google.com/p/epubcheck>`_.

If you plan to make PDF or mathematical formulas via LaTeX, you will need:

.. code-block:: bash

    $ sudo aptitude install dvipng
    $ sudo aptitude install texlive texlive-luatex texlive-xetex texlive-latex-extra texlive-fonts-extra
    $ sudo aptitude install texlive-lang-french

In the last command, change `french` with your language.


.. _installation_deployment_database:

Setting up the Database
-----------------------

You need to create the database that matches your future configuration.

For instance, if you decide to use PostgreSQL (the recommended database),
invoke something like:

.. code-block:: bash

    $ sudo aptitude install postgresql
    $ sudo su - postgres -c "createuser --no-superuser --pwprompt publiforge"
    $ sudo su - postgres -c "createdb -T template0 -E UTF-8 -O publiforge PFInstance"

The default password is `sesame`.
 
If you decide to use MySQL, install the follwing Python package:

.. code-block:: bash

    $ source /usr/local/virtualenv/bin/activate
    (virtualenv)$ pip install MySQL-python

Then, install MySQL:

.. code-block:: bash

    $ sudo aptitude install mysql-server
    $ mysql -u root -p mysql
    mysql> GRANT SELECT,INSERT,UPDATE,DELETE,INDEX,CREATE,DROP ON *.* TO publiforge@localhost IDENTIFIED BY 'sesame';
    mysql> CREATE DATABASE PubliForge;
    mysql> QUIT;


Installing `PubliForge`
-----------------------

Now, it is time to check out sources and install them in your virtual
environment:

.. code-block:: bash

    $ source /usr/local/virtualenv/bin/activate
    (virtualenv)$ pip install Mercurial
    (virtualenv)$ hg clone https://hg.publiforge.org/PubliForge
    (virtualenv)$ cd PubliForge
    (virtualenv)$ pip install .

You can also use ``pip install -e .`` instead of ``pip install .`` to keep the
link between your installation and `PubliForge` sources. In that case, you can
create the cached byte-code files:

.. code-block:: bash

    (virtualenv)$ python -m compileall publiforge/


Setting Up `PubliForge` Environment
-----------------------------------

Within ``pfinstance`` home directory, make the following structure:

.. code-block:: bash

    $ su pfinstance
    $ cd
    $ mkdir PubliForge
    $ cd PubliForge
    $ mkdir Cache Storages Builds Buildspaces

And create a ``pfinstance.ini`` (or a ``production.ini``) file like the
following:

.. code-block:: ini

    # =============================================================================
    [server:main]
    use = egg:waitress#main
    host = 127.0.0.1
    port = %(http_port)s
    #numthreads = 10
    #timeout = 180
    #request_queue_size = 200

    # =============================================================================
    [app:main]
    use = egg:PubliForge

    # ------ Instance defintion
    uid = pfinstance
    encryption = secretkey
    #maintenance = true

    # ------ Internationalization
    languages = fr, en
    #pyramid.default_locale_name = en
    #translation_dirs =

    # ------ Debug mode off
    pyramid.reload_templates = false
    pyramid.debug_authorization = false
    pyramid.debug_notfound = false
    pyramid.debug_routematch = false
    pyramid.debug_templates = false

    # ------------------------------ Front

    # ------ Skin
    skin.name = pfinstance
    skin.title = PubliForge – PFInstance
    #skin.static.name =
    #skin.static.path =
    #skin.favicon =
    #skin.css.base =
    #skin.css.login =
    #skin.css.custom =
    #skin.template.base =
    #skin.template.login =

    # ------ Temporary directory
    temporary_dir = %(here)s/Cache/Tmp

    # ------ Beaker cache
    #cache.type = file
    #cache.data_dir = %(here)s/Cache/BeakerCache/Data
    #cache.lock_dir = %(here)s/Cache/BeakerCache/Lock

    # ------ Beaker session
    #session.type = file
    #session.data_dir = %(here)s/Cache/BeakerSessions/Data
    #session.lock_dir = %(here)s/Cache/BeakerSessions/Lock
    session.key = PF_SESSION

    # ------ SQLAlchemy
    # ... PostgreSQL
    sqlalchemy.url = postgresql://publiforge:sesame@localhost/PFInstance
    #sqlalchemy.pool_size = 5
    #sqlalchemy.max_overflow = 10
    #sqlalchemy.pool_timeout = 30
    # ... MySQL
    #sqlalchemy.url = mysql://publiforge:sesame@localhost/PFInstance?charset=utf8
    #sqlalchemy.pool_recycle = 3600

    # ------ Authentication
    #auth.cookie = PF_AUTH
    #auth.remember = 5184000

    # ------ Page refresh
    #refresh.short = 2
    #refresh.long = 5

    # ------ Storages
    storage.root = %(here)s/Storages
    storage.index = %(here)s/StoragesIndex
    storage.vcs = none, local, hg, hgsvn
    #storage.report_ttl = 120
    #storage.cache = 3600

    # ------ File openers
    opener.roots = %(here)s/Openers
    opener.list = Publi*, Ini*
    opener.cache = %(here)s/Cache/Openers

    # ------ Available agents
    #agent.refresh = 0
    #agent.synchronize = 1
    agent.0.weight = 1
    agent.0.password = sesame

    # ------------------------------ Agent

    # ------ Buildspaces
    buildspace.root = %(here)s/Buildspaces
    #buildspace.ttl = 2678400

    # ------ Processors
    processor.roots = %(here)s/Processors
    processor.list = Publi*, PackFilling, Parallel, CharCount, DTBookValid

    # ------ Builds
    #build.reset = false
    build.root = %(here)s/Builds
    #build.ttl = 1800
    #build.result_ttl = 604800
    #build.concurrent = 3

    # ------ Authorized fronts
    front.0.uid = pfinstance
    front.0.password = sesame

    # =============================================================================
    [Populate]
    # ------ Administrator
    admin.login = admin
    admin.password = adminpwd
    admin.name = Administrateur
    admin.email = admin@prismallia.fr

    # =============================================================================
    # ====== Logging configuration
    [loggers]
    keys = root, publiforge, sqlalchemy, activity

    [handlers]
    keys = console, activity

    [formatters]
    keys = generic, filelog

    # ====== Loggers
    [logger_root]
    level = WARN
    handlers = console

    [logger_publiforge]
    level = WARN
    handlers =
    qualname = publiforge

    [logger_sqlalchemy]
    level = WARN
    handlers =
    qualname = sqlalchemy.engine

    [logger_activity]
    level = INFO
    handlers = activity
    qualname = activity
    propagate = 0

    # ====== Handlers
    [handler_console]
    class = StreamHandler
    args = (sys.stderr,)
    level = NOTSET
    formatter = generic

    [handler_activity]
    class=handlers.TimedRotatingFileHandler
    args=('%(here)s/Log/publiforge.log', 'w6', 1, 52, 'utf-8', True)
    level=NOTSET
    formatter=filelog

    # ====== Formatters
    [formatter_generic]
    format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s

    [formatter_filelog]
    format = %(asctime)s %(message)s

You can read :ref:`installation_customization` for a precise customization.


Populating the Database and the Storages
----------------------------------------

Use the :ref:`frontguide_scripts_pfpopulate` console script to populate your
database and your storages. To do so, type the following command:

.. code-block:: bash

    $ source /usr/local/virtualenv/bin/activate
    (virtualenv)$ pfpopulate pfinstance.ini


.. _installation_deployment_scheduler:

Activating Job Scheduler
------------------------

A ``background job`` is a processing which is periodically run by a
scheduler. If you plan to use this functionality, you have to configure the
CRON table of the PubliForge user. To do so, execute the following command:

.. code-block:: bash

    $ su pfinstance
    pfinstance$ crontab -e

And enter a configuration such as:

.. code-block:: bash

    */6 * * * *  /usr/local/virtualenv/bin/pfscheduler ~/PubliForge/pfinstance.ini

Now, every 6 minutes, the PubliForge scheduler will run the pending jobs.


Deploying
---------

You can now deploy your instance with `Apache <http://httpd.apache.org>`_ (see
:ref:`installation_deployment_apache`) or `Nginx <http://nginx.org/en>`_
(see :ref:`installation_deployment_nginx`).