[PATCH 00/10] Toaster documentation update

[PATCH 00/10] Toaster documentation update

[Ed Bartosh]

Hi,

This patchset contains changes to Toaster documentation caused by the switch
to Python 3:
 - usage of pip3 --user in favor of virtualenv
 - usage of mod_wsgi for python 3
 - changed list of required packages

There are also changes unrelated to Python 3:
 - added instructions for production setup on OpenSUSE Leap 42.1
 - changed instructions for apache as it's upgraded to 2.4 in the latest distro versions
 - used isolated setup for runbuilds using screen/tmux terminal multiplexers

The setup was tested mainly on OpenSUSE leap 42.1 host and partly in docker containers for
Ubuntu 16.04 LTS and Fedora 24.

Note: While working on this and looking at new ways of delivering software opened by
Docker I tend to think that we should start building preconfigured docker containers for
Toaster instead of maintaining setup and deployment instructions.

The following changes since commit d9c6c6e1a241913fae26bf97030965ef89b91b0f:

  toaster: remove SSTATE_MIRRORS from projectconf (2016-06-24 16:05:19 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/toaster/python3-documentation
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/python3-documentation

Ed Bartosh (9):
  toaster-manual: update distro versions
  toaster-manual: update list of packages
  toaster-manual: add list of packages for openSUSE
  toaster-manual: add instructions to create user
  toaster-manual: change mysql driver
  toaster-manual: add instructions for OpenSUSE
  toaster-manual: update apache configuration file
  toaster-manual: update runbuilds instructions
  toaster-manual: add OpenSUSE to the list of distros

brian avery (1):
  toaster-manual: Switch to  pip3 dependency installer

 .../toaster-manual-setup-and-use.xml               | 95 +++++++++++++---------
 .../toaster-manual/toaster-manual-start.xml        | 72 +++++-----------
 2 files changed, 77 insertions(+), 90 deletions(-)

--
Regards,
Ed

[PATCH 01/10] toaster-manual: Switch to pip3 dependency installer

[Ed Bartosh]

From: brian avery <brian.avery@intel.com>

Our switch to python3 requires us to dump the virtualenv approach
as long as we need python to resolve to python2 in the same shell
that python3 resolves to python3.  To enable users to install
Toaster's python package dependencies locally, we are
changing to using pip3 and these doc changes explain how to do
that.

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 .../toaster-manual-setup-and-use.xml               | 26 ++++----
 .../toaster-manual/toaster-manual-start.xml        | 72 ++++++----------------
 2 files changed, 31 insertions(+), 67 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 963b211..6abbcb7 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -116,12 +116,12 @@
             create a superuser by following these steps:
             <orderedlist>
               <listitem><para>
-                  If you used <filename>virtualenv</filename>, which is
+                  If you used <filename>pip3</filename>, which is
                   recommended, to set up the Toaster system dependencies,
-                  you need be sure the virtual environment is activated.
-                  To activate this environment, use the following command:
+                  you need be sure the local user path is in your PATH list.
+                  To append the pip3 local user path, use the following command:
                   <literallayout class='monospaced'>
-   $ source venv/bin/activate
+   $ export PATH=$PATH:$HOME/.local/bin
                   </literallayout>
                   </para></listitem>
               <listitem><para>
@@ -213,14 +213,14 @@
                   <listitem><para>
                       If you are using Ubuntu 14.04.3, run the following:
                       <literallayout class='monospaced'>
-   $ sudo apt-get install apache2 libapache2-mod-wsgi mysql-server virtualenv libmysqlclient-dev
+   $ sudo apt-get install apache2 libapache2-mod-wsgi mysql-server python3-pip libmysqlclient-dev
                       </literallayout>
                       </para></listitem>
                   <listitem><para>
                       If you are using Fedora 22 or a RedHat distribution, run
                       the following:
                       <literallayout class='monospaced'>
-   $ sudo dnf install httpd mod_wsgi python-virtualenv gcc mysql-devel
+   $ sudo dnf install httpd mod_wsgi  gcc mysql-devel
                       </literallayout>
                       </para></listitem>
               </itemizedlist>
@@ -245,17 +245,15 @@
                       </literallayout>
                       </para></listitem>
                   <listitem><para>
-                      Initialize a virtual environment and install Toaster
-                      dependencies.
-                      Using a virtual environment keeps the Python packages
+                      Install Toaster
+                      dependencies using the --user flag which
+                      keeps the Python packages
                       isolated from your system-provided packages:
                       <literallayout class='monospaced'>
    $ cd /var/www/toaster/
-   $ virtualenv venv
-   $ source ./venv/bin/activate
-   $ pip install -r ./poky/bitbake/toaster-requirements.txt
-   $ pip install mysql
-   $ pip install MySQL-python
+   $ pip3 install --user -r ./poky/bitbake/toaster-requirements.txt
+   $ pip3 install --user mysql
+   $ pip3 install --user MySQL-python
                       </literallayout>
                       <note>
                           Isolating these packages is not required but is
diff --git a/documentation/toaster-manual/toaster-manual-start.xml b/documentation/toaster-manual/toaster-manual-start.xml
index daefa79..71f9151 100644
--- a/documentation/toaster-manual/toaster-manual-start.xml
+++ b/documentation/toaster-manual/toaster-manual-start.xml
@@ -22,6 +22,11 @@
             and
             "<ulink url='&YOCTO_DOCS_QS_URL;#releases'>Yocto Project Release</ulink>"
             sections in the Yocto Project Quick Start.
+	    For Ubuntu/Debian you may need to do an additional install of pip3.
+                <literallayout class='monospaced'>
+     $ sudo apt-get install python3-pip
+                </literallayout>
+
         </para>
     </section>
 
@@ -42,56 +47,6 @@
             install-compatible format.
         </para>
 
-        <section id='toaster-virtual-environment'>
-            <title>Set Up a Python Virtual Environment</title>
-
-            <para>
-                Set up a Python virtual environment that allows you
-                to maintain a dedicated Python executable and its own
-                set of installed modules.
-                Doing so separates the executable from Python and the
-                modules provided by the operating system.
-                This separation avoids any version conflicts.
-                <note>
-                    Creating a virtual environment is not absolutely
-                    necessary.
-                    However, doing so is highly recommended.
-                </note>
-            </para>
-
-            <para>
-                Follow these steps to set up your virtual environment.
-                These steps assume a Ubuntu distribution:
-                <orderedlist>
-                    <listitem><para><emphasis>Install <filename>virtualenv</filename>:</emphasis>
-                        Install the supported
-                        <filename>python-virtualenv</filename> package from your
-                        distribution rather than using <filename>pip</filename>.
-                        <literallayout class='monospaced'>
-     $ sudo apt-get install python-virtualenv
-                        </literallayout>
-                        </para></listitem>
-                    <listitem><para><emphasis>Create and Activate a Virtual Environment:</emphasis>
-                        <literallayout class='monospaced'>
-     $ virtualenv venv
-     $ source venv/bin/activate
-                        </literallayout>
-                        </para></listitem>
-                </orderedlist>
-                <note>
-                    After setting up a virtual environment in
-                    which to run Toaster, you must initialize that
-                    virtual environment each time you want to start
-                    Toaster.
-                    Use the following to initialize the environment just
-                    before you start Toaster:
-                    <literallayout class='monospaced'>
-     $ source venv/bin/activate
-                    </literallayout>
-                </note>
-            </para>
-        </section>
-
         <section id='toaster-load-packages'>
             <title>Install Toaster Packages</title>
 
@@ -99,9 +54,20 @@
                 You need to install the packages that Toaster requires.
                 Use this command:
                 <literallayout class='monospaced'>
-     $ pip install -r bitbake/toaster-requirements.txt
+     $ pip3 install --user -r bitbake/toaster-requirements.txt
                 </literallayout>
             </para>
+	    <para>
+	      This will install the necessary Toaster modules into a local python 3 cache in your $HOME directory.  The caches is actually located in $HOME/.local  To see what packages have been installed into your $HOME direcotry you can do:
+                <literallayout class='monospaced'>
+     $ pip3 list installed --local
+                </literallayout>
+And, if you need to remove something the following will work:
+                <literallayout class='monospaced'>
+     $ pip3 uninstall PackageNameToUninstall
+                </literallayout>
+	    </para>
+
         </section>
 
 <!-- Commenting this section out for now in case it needs to be used again.
@@ -120,7 +86,7 @@
                         install <filename>daemon</filename> using the
                         following command:
                         <literallayout class='monospaced'>
-     $ sudo apt-get install daemon​
+     $ sudo apt-get install daemon
                         </literallayout>
                         </para></listitem>
                     <listitem><para><emphasis>Non-Debian-Based Systems:</emphasis>
@@ -137,7 +103,7 @@
                         Here are the commands for a 32-bit distribution:
                         <literallayout class='monospaced'>
      $ wget http://libslack.org/daemon/download/daemon-0.6.4-1.i686.rpm
-     $ sudo rpm -i ​daemon-0.6.4-1.i686.rpm​
+     $ sudo rpm -i ​daemon-0.6.4-1.i686.rpm
                         </literallayout>
                         </para></listitem>
                 </itemizedlist>
-- 
2.6.6

[PATCH 02/10] toaster-manual: update distro versions

[Ed Bartosh]

Changed Ubuntu and Fedora versions to the latest ones.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 6abbcb7..2abf05f 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -211,13 +211,13 @@
                       Use the Mysql database server.
                       </para></listitem>
                   <listitem><para>
-                      If you are using Ubuntu 14.04.3, run the following:
+                      If you are using Ubuntu 16.04, run the following:
                       <literallayout class='monospaced'>
    $ sudo apt-get install apache2 libapache2-mod-wsgi mysql-server python3-pip libmysqlclient-dev
                       </literallayout>
                       </para></listitem>
                   <listitem><para>
-                      If you are using Fedora 22 or a RedHat distribution, run
+                      If you are using Fedora 24 or a RedHat distribution, run
                       the following:
                       <literallayout class='monospaced'>
    $ sudo dnf install httpd mod_wsgi  gcc mysql-devel
-- 
2.6.6

[PATCH 03/10] toaster-manual: update list of packages

[Ed Bartosh]

Changed mod_wsgi package names as Toaster requires python 3 version of
mod_wsgi.

Added missing python3-pip, mariadb-server, mariadb-devel and python3-devel
to the list of packages for Fedora.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 2abf05f..3cc41bb 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -213,14 +213,14 @@
                   <listitem><para>
                       If you are using Ubuntu 16.04, run the following:
                       <literallayout class='monospaced'>
-   $ sudo apt-get install apache2 libapache2-mod-wsgi mysql-server python3-pip libmysqlclient-dev
+   $ sudo apt-get install apache2 libapache2-mod-wsgi-py3 mysql-server python3-pip libmysqlclient-dev
                       </literallayout>
                       </para></listitem>
                   <listitem><para>
                       If you are using Fedora 24 or a RedHat distribution, run
                       the following:
                       <literallayout class='monospaced'>
-   $ sudo dnf install httpd mod_wsgi  gcc mysql-devel
+   $ sudo dnf install httpd python3-mod_wsgi python3-pip mariadb-server mariadb-devel python3-devel
                       </literallayout>
                       </para></listitem>
               </itemizedlist>
-- 
2.6.6

[PATCH 04/10] toaster-manual: add list of packages for openSUSE

[Ed Bartosh]

Added package installation instructions for openSUSE Leap 42.1

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 3cc41bb..2e085a2 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -223,6 +223,13 @@
    $ sudo dnf install httpd python3-mod_wsgi python3-pip mariadb-server mariadb-devel python3-devel
                       </literallayout>
                       </para></listitem>
+                  <listitem><para>
+                      If you are using openSUSE Leap 42.1, run
+                      the following:
+                      <literallayout class='monospaced'>
+   $ sudo zypper install apache2 apache2-mod_wsgi-python3 python3-pip mariadb mariadb-client python3-devel
+                      </literallayout>
+                      </para></listitem>
               </itemizedlist>
           </para>
       </section>
-- 
2.6.6

[PATCH 05/10] toaster-manual: add instructions to create user

[Ed Bartosh]

Creating separate toaster account would make production setup more
isolated and maintainable.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 2e085a2..4349eb1 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -241,12 +241,18 @@
               Perform the following steps to install Toaster:
               <orderedlist>
                   <listitem><para>
+                      Create toaster user, set its home directory to /var/www/toaster and switch to it:
+                      <literallayout class='monospaced'>
+   $ sudo /usr/sbin/useradd toaster -d /var/www/toaster -s /bin/false
+   $ sudo mkdir /var/www/toaster
+   $ sudo su - toaster -s /bin/bash
+                      </literallayout>
+                  </para></listitem>
+                  <listitem><para>
                       Checkout a copy of <filename>poky</filename>
                       into the web server directory.
                       You will be using <filename>/var/www/toaster</filename>:
                       <literallayout class='monospaced'>
-   $ mkdir -p /var/www/toaster
-   $ cd /var/www/toaster/
    $ git clone git://git.yoctoproject.org/poky
    $ git checkout &DISTRO_NAME_NO_CAP;
                       </literallayout>
-- 
2.6.6

[PATCH 06/10] toaster-manual: change mysql driver

[Ed Bartosh]

Changed mysql driver to mysqlclient as MySQLdb doesn't
support Python 3 yet.

Here is a quote from Django 1.9 documentation regarding this:
mysqlclient is a fork of MySQLdb which notably supports
Python 3 and can be used as a drop-in replacement for
MySQLdb. At the time of this writing, this is the
recommended choice for using MySQL with Django.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 4349eb1..1c9be88 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -265,8 +265,7 @@
                       <literallayout class='monospaced'>
    $ cd /var/www/toaster/
    $ pip3 install --user -r ./poky/bitbake/toaster-requirements.txt
-   $ pip3 install --user mysql
-   $ pip3 install --user MySQL-python
+   $ pip3 install --user mysqlclient
                       </literallayout>
                       <note>
                           Isolating these packages is not required but is
-- 
2.6.6

[PATCH 07/10] toaster-manual: add instructions for OpenSUSE

[Ed Bartosh]

Added description of the location of apache configuration
file for OpenSUSE Leap 42.1

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 1c9be88..5cc2595 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -388,6 +388,10 @@
                       <literallayout class='monospaced'>
    /etc/httpd/conf.d/toaster.conf
                       </literallayout>
+                      If you are using OpenSUSE, put it here:
+                      <literallayout class='monospaced'>
+   /etc/apache2/conf.d/toaster.conf
+                      </literallayout>
                       Following is a sample Apache configuration for Toaster
                       you can follow:
                       <literallayout class='monospaced'>
-- 
2.6.6

[PATCH 08/10] toaster-manual: update apache configuration file

[Ed Bartosh]

Changed access control instructions to work with Apache 2.4
according to https://httpd.apache.org/docs/current/upgrading.html

Added 'Require all granted' for wsgi.py according to
https://docs.djangoproject.com/en/1.9/howto/deployment/wsgi/modwsgi/

Changed python module path for WSGIDaemonProcess as python version
changed and modules are installed to the different path.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 .../toaster-manual/toaster-manual-setup-and-use.xml  | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 5cc2595..febbd15 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -397,16 +397,26 @@
                       <literallayout class='monospaced'>
    Alias /static /var/www/toaster/static_files
    &lt;Directory /var/www/toaster/static_files&gt;
-           Order allow,deny
-           Allow from all
-           Require all granted
+           &lt;IfModule mod_access_compat.c&gt;
+                   Order allow,deny
+                   Allow from all
+           &lt;/IfModule&gt;
+           &lt;IfModule !mod_access_compat.c&gt;
+                   Require all granted
+           &lt;/IfModule&gt;
    &lt;/Directory&gt;
 
-   WSGIDaemonProcess toaster_wsgi python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/venv/lib/python2.7/site-packages
+   &lt;Directory /var/www/toaster/poky/bitbake/lib/toaster/toastermain&gt;
+           &lt;Files "wsgi.py"&gt;
+              Require all granted
+           &lt;/Files&gt;
+   &lt;/Directory&gt;
+
+   WSGIDaemonProcess toaster_wsgi python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/.local/lib/python3.4/site-packages
 
    WSGIScriptAlias / "/var/www/toaster/poky/bitbake/lib/toaster/toastermain/wsgi.py"
    &lt;Location /&gt;
-       WSGIProcessGroup toastern_wsgi
+       WSGIProcessGroup toaster_wsgi
    &lt;/Location&gt;
                       </literallayout>
                       If you are using Ubuntu or Debian,
-- 
2.6.6

[PATCH 09/10] toaster-manual: update runbuilds instructions

[Ed Bartosh]

As Toaster runbuild heavily depends on the environment set by
oe-init-build-env and toaster script, running runbuilds is not
enough anymore. Both above mentioned scripts shoud be sourced
in isolated shell environment using screen or tmux terminal
multiplexers to make runbuilds service working.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 .../toaster-manual-setup-and-use.xml               | 23 ++++++++++------------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index febbd15..c37fa65 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -438,20 +438,17 @@
                       </literallayout>
                       </para></listitem>
                   <listitem><para>
-                      Install the build runner service.
-                      This service needs to be running in order to dispatch
-                      builds.
-                      Use this command:
+                      Run Toaster script with 'noweb' command line option.
+                      This needs to be done in order to dispatch builds.
+                      Note that this is better to do in screen or tmux session to be able to detach from it
+                      without stopping Toaster from dispatching builds.
+                      Use these commands:
                       <literallayout class='monospaced'>
-   /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
-                      </literallayout>
-                      Here is an example:
-                      <literallayout class='monospaced'>
-   #!/bin/sh
-   # toaster run builds dispatcher
-   cd /var/www/toaster/
-   source ./venv/bin/activate
-   ./bitbake/lib/toaster/manage.py runbuilds
+   $ sudo su - toaster -s /bin/bash
+   $ screen
+   $ cd poky
+   $ . oe-init-build-env
+   $ . ../bitbake/bin/toaster start noweb
                       </literallayout>
                       </para></listitem>
               </orderedlist>
-- 
2.6.6

[PATCH 10/10] toaster-manual: add OpenSUSE to the list of distros

[Ed Bartosh]

Updated instructions of restarting apache. As OpenSUSE way
of restarting apache is the same as Ubuntu and Debian it's
enough to just add OpenSUSE to the list of distros.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 documentation/toaster-manual/toaster-manual-setup-and-use.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index c37fa65..d0d37d3 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -428,7 +428,7 @@
                       </literallayout>
                       Finally, restart Apache to make sure all new configuration
                       is loaded.
-                      For Ubuntu and Debian use:
+                      For Ubuntu, Debian and OpenSUSE use:
                       <literallayout class='monospaced'>
    $ sudo service apache2 restart
                       </literallayout>
-- 
2.6.6

Re: [PATCH 09/10] toaster-manual: update runbuilds instructions

[Michael Wood]

On 27/06/16 16:24, Ed Bartosh wrote:
> As Toaster runbuild heavily depends on the environment set by
> oe-init-build-env and toaster script, running runbuilds is not
> enough anymore. Both above mentioned scripts shoud be sourced
> in isolated shell environment using screen or tmux terminal
> multiplexers to make runbuilds service working.
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>   .../toaster-manual-setup-and-use.xml               | 23 ++++++++++------------
>   1 file changed, 10 insertions(+), 13 deletions(-)
>
> diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> index febbd15..c37fa65 100644
> --- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> +++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> @@ -438,20 +438,17 @@
>                         </literallayout>
>                         </para></listitem>
>                     <listitem><para>
> -                      Install the build runner service.
> -                      This service needs to be running in order to dispatch
> -                      builds.
> -                      Use this command:
> +                      Run Toaster script with 'noweb' command line option.
> +                      This needs to be done in order to dispatch builds.
> +                      Note that this is better to do in screen or tmux session to be able to detach from it
> +                      without stopping Toaster from dispatching builds.
> +                      Use these commands:
>                         <literallayout class='monospaced'>
> -   /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
> -                      </literallayout>
> -                      Here is an example:
> -                      <literallayout class='monospaced'>
> -   #!/bin/sh
> -   # toaster run builds dispatcher
> -   cd /var/www/toaster/
> -   source ./venv/bin/activate
> -   ./bitbake/lib/toaster/manage.py runbuilds
> +   $ sudo su - toaster -s /bin/bash
> +   $ screen
> +   $ cd poky
> +   $ . oe-init-build-env
> +   $ . ../bitbake/bin/toaster start noweb
>                         </literallayout>
>                         </para></listitem>
>                 </orderedlist>

We can't really suggest that people use screen or tmux for a production 
system, as that requires manual intervention to start and stop it. The 
user which they're running this as might not actually be a full blown 
system user either and we wouldn't recommend it being one. A better 
approach would be to write a systemd or sysvint, upstart etc service. e.g.

author "Michael W"
description "start and stop toaster-next build service"
version "1.0"

start on started networking
stop on runlevel [!2345]

respawn

script
   exec su toasterbuilder -c source oe-init-build-env && export 
DJANGO_SETTINGS_MODULE=toastermain.settings-production && source toaster 
start noweb

end script

Re: [PATCH 09/10] toaster-manual: update runbuilds instructions

[Ed Bartosh]

On Tue, Jun 28, 2016 at 04:41:34PM +0100, Michael Wood wrote:
> On 27/06/16 16:24, Ed Bartosh wrote:
> >As Toaster runbuild heavily depends on the environment set by
> >oe-init-build-env and toaster script, running runbuilds is not
> >enough anymore. Both above mentioned scripts shoud be sourced
> >in isolated shell environment using screen or tmux terminal
> >multiplexers to make runbuilds service working.
> >
> >Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> >---
> >  .../toaster-manual-setup-and-use.xml               | 23 ++++++++++------------
> >  1 file changed, 10 insertions(+), 13 deletions(-)
> >
> >diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> >index febbd15..c37fa65 100644
> >--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> >+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> >@@ -438,20 +438,17 @@
> >                        </literallayout>
> >                        </para></listitem>
> >                    <listitem><para>
> >-                      Install the build runner service.
> >-                      This service needs to be running in order to dispatch
> >-                      builds.
> >-                      Use this command:
> >+                      Run Toaster script with 'noweb' command line option.
> >+                      This needs to be done in order to dispatch builds.
> >+                      Note that this is better to do in screen or tmux session to be able to detach from it
> >+                      without stopping Toaster from dispatching builds.
> >+                      Use these commands:
> >                        <literallayout class='monospaced'>
> >-   /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
> >-                      </literallayout>
> >-                      Here is an example:
> >-                      <literallayout class='monospaced'>
> >-   #!/bin/sh
> >-   # toaster run builds dispatcher
> >-   cd /var/www/toaster/
> >-   source ./venv/bin/activate
> >-   ./bitbake/lib/toaster/manage.py runbuilds
> >+   $ sudo su - toaster -s /bin/bash
> >+   $ screen
> >+   $ cd poky
> >+   $ . oe-init-build-env
> >+   $ . ../bitbake/bin/toaster start noweb
> >                        </literallayout>
> >                        </para></listitem>
> >                </orderedlist>
> 
> We can't really suggest that people use screen or tmux for a
> production system, as that requires manual intervention to start and
> stop it. The user which they're running this as might not actually
> be a full blown system user either and we wouldn't recommend it
> being one. A better approach would be to write a systemd or sysvint,
> upstart etc service. e.g.
> 
> author "Michael W"
> description "start and stop toaster-next build service"
> version "1.0"
> 
> start on started networking
> stop on runlevel [!2345]
> 
> respawn
> 
> script
>   exec su toasterbuilder -c source oe-init-build-env && export
> DJANGO_SETTINGS_MODULE=toastermain.settings-production && source
> toaster start noweb
> 
> end script

Thank you for the suggestions! I've added explanation of how to create
and run systemd service. Please, review my branch ed/toaster/python3-documentation

--
Regards,
Ed

Re: [PATCH 09/10] toaster-manual: update runbuilds instructions

[Michael Wood]

Pasting this in as it's an updated version of this patch:


commit 74e368b1855bbf4321f73b09a0f2c973b1e33c61
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date:   Mon Jun 27 17:37:40 2016 +0300

     toaster-manual: update runbuilds instructions

     Added sample configuration files and instructions for the runbuilds
     systemd service. Service runs runbuilds in a screen session for user
     convenience. User can attach to the session to see the runbuilds 
output,
     monitor logs and build environment.

     Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git 
a/documentation/toaster-manual/toaster-manual-setup-and-use.xml 
b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index febbd15..26e71c8 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -438,22 +438,49 @@
                        </literallayout>
                        </para></listitem>
                    <listitem><para>
-                      Install the build runner service.
-                      This service needs to be running in order to dispatch
-                      builds.
-                      Use this command:
+                      Prepare systemd service to run toaster builds.
+                      Here is a sample configuration file for the service:
                        <literallayout class='monospaced'>
-   /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
+   [Unit]
+   Description=Toaster runbuilds
+
+   [Service]
+   Type=forking
+   User=toaster
+   ExecStart=/usr/bin/screen -d -m -S runbuilds 
/var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start
+   ExecStop=/usr/bin/screen -S runbuilds -X quit
+   WorkingDirectory=/var/www/toaster/poky
+


This is kind of interesting from a developer of Toaster point of view 
but I don't think we should be recommending screen in the production 
setup. Toaster should be logging all the useful output anyway.



+   [Install]
+   WantedBy=multi-user.target
                        </literallayout>
-                      Here is an example:
+                      And the runbuilds-service.sh script that you 
should put into /var/www/toaster/poky/bitbake/lib/toaster/ and
+                      set executable permissions to it.
                        <literallayout class='monospaced'>
-   #!/bin/sh
-   # toaster run builds dispatcher
-   cd /var/www/toaster/
-   source ./venv/bin/activate
-   ./bitbake/lib/toaster/manage.py runbuilds
+   #!/bin/bash
+
+   #export http_proxy=http://proxy.host.com:8080
+   #export https_proxy=http://proxy.host.com:8080
+   #export GIT_PROXY_COMMAND=$HOME/bin/gitproxy
+
+   cd ~/poky/
+   source ./oe-init-build-env build
+   source ../bitbake/bin/toaster $1 noweb
+   [ "$1" == 'start' ] &amp;&amp; /bin/bash
                        </literallayout>
-                      </para></listitem>
+                  </para></listitem>
+                  <listitem><para>
+                      Run the service:
+                      <literallayout class='monospaced'>
+   # service runbuilds start
+                      </literallayout>
+                      As service is running in a detached screen 
session you can attach to it using this command:
+                      <literallayout class='monospaced'>
+   $ sudo su - toaster
+   $ screen -rS runbuilds
+                      </literallayout>
+                      And detach from it again using 'Ctrl-a' followed 
by 'd' key combination.
+                  </para></listitem>
                </orderedlist>
                You can now open up a browser and start using Toaster.
            </para>

[PATCH] toaster-manual: Update toaster start documentation

[Michael Wood]

- Update starting toaster to source toaster start
- Add webport address documentation
- Remove redundant "syncdb" call
- Make it clearer that collectstatic uses STATIC_ROOT

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
---
 .../toaster-manual-setup-and-use.xml               | 24 +++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 66a95ac..1a87b49 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -31,7 +31,7 @@
             <filename>poky/build</filename>), start Toaster using this
             command:
             <literallayout class='monospaced'>
-     $ source ../bitbake/bin/toaster
+     $ source toaster start
             </literallayout>
             You can now run your builds from the command line, or with
             Toaster as explained in section
@@ -56,7 +56,21 @@
             set a different port.
             For example, the following command sets the port to "8400":
             <literallayout class='monospaced'>
-     $ source ../bitbake/bin/toaster webport=8400
+     $ source toaster start webport=8400
+            </literallayout>
+        </para>
+    </section>
+
+    <section id='setting-a-different-address'>
+        <title>Setting a Different Address</title>
+
+        <para>
+            By default, Toaster binds to the loop back address i.e. localhost.
+            You can use the <filename>WEBPORT</filename> parameter to
+            set a different host.
+            For example, the following command sets the host and port to "192.168.0.1:8400":
+            <literallayout class='monospaced'>
+     $ source toaster start webport=192.168.0.1:8400
             </literallayout>
         </para>
     </section>
@@ -324,7 +338,6 @@
                       default data, and gather the statically-served files:
                       <literallayout class='monospaced'>
    $ cd  /var/www/toaster/poky/
-   $ ./bitbake/lib/toaster/manage.py syncdb
    $ ./bitbake/lib/toaster/manage.py migrate
    $ TOASTER_DIR=`pwd` TOASTER_CONF=./meta-poky/conf/toasterconf.json ./bitbake/lib/toaster/manage.py checksettings
    $ ./bitbake/lib/toaster/manage.py collectstatic
@@ -334,7 +347,7 @@
                       <para>
                           For the above set of commands, after moving to the
                           <filename>poky</filename> directory,
-                          the <filename>syncdb</filename> and <filename>migrate</filename>
+                          the <filename>migrate</filename>
                           commands ensure the database
                           schema has had changes propagated correctly (i.e.
                           migrations).
@@ -375,7 +388,8 @@
                           Finally, the <filename>collectstatic</filename> command
                           is a Django framework command that collects all the
                           statically served files into a designated directory to
-                          be served up by the Apache web server.
+                          be served up by the Apache web server as defined by
+                          <filename>STATIC_ROOT</filename>
                       </para></listitem>
                   <listitem><para>
                       Add an Apache configuration file for Toaster to your Apache web
-- 
2.7.4

Re: [PATCH 09/10] toaster-manual: update runbuilds instructions

[Brian Avery]

[-- Attachment #1: Type: text/plain, Size: 5248 bytes --]

Hi,

I was trying out your instructions/production setup on an ubuntu
16.04/mysql setup and got the following error:
$>TOASTER_DIR=`pwd` TOASTER_CONF=./meta-poky/conf/toasterconf.json
./bitbake/lib/toaster/manage.py checksettings --traceback 2>&1 | tee
mychecksettings.log

resulted in

django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1")

I'm attaching the logfile.

Have any of you seen this before? It kinda stops me cold here. Sadly,
the error doesn't let me know what table or data was too long.

-b
an intel employee

p.s. other issues I've seen so far
1) 3.7.2.1
sudo mkdir /var/www/toaster
need a : sudo chown toaster /var/www/toaster
2) 3.7.2.4
SECRET_KEY needs a pointer to the Django documentation so people know
what it is:
https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SECRET_KEY

3) 3.7.2.6
the TOASTER_DIR=... checksettings line is too long and got truncated
for me when my pdf was made.  We should probably break it in 2 or use
exports.






On Fri, Jul 8, 2016 at 6:30 AM, Michael Wood <michael.g.wood@intel.com> wrote:
> Pasting this in as it's an updated version of this patch:
>
>
> commit 74e368b1855bbf4321f73b09a0f2c973b1e33c61
> Author: Ed Bartosh <ed.bartosh@linux.intel.com>
> Date:   Mon Jun 27 17:37:40 2016 +0300
>
>     toaster-manual: update runbuilds instructions
>
>     Added sample configuration files and instructions for the runbuilds
>     systemd service. Service runs runbuilds in a screen session for user
>     convenience. User can attach to the session to see the runbuilds output,
>     monitor logs and build environment.
>
>     Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
>
> diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> index febbd15..26e71c8 100644
> --- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> +++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> @@ -438,22 +438,49 @@
>                        </literallayout>
>                        </para></listitem>
>                    <listitem><para>
> -                      Install the build runner service.
> -                      This service needs to be running in order to dispatch
> -                      builds.
> -                      Use this command:
> +                      Prepare systemd service to run toaster builds.
> +                      Here is a sample configuration file for the service:
>                        <literallayout class='monospaced'>
> -   /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
> +   [Unit]
> +   Description=Toaster runbuilds
> +
> +   [Service]
> +   Type=forking
> +   User=toaster
> +   ExecStart=/usr/bin/screen -d -m -S runbuilds
> /var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start
> +   ExecStop=/usr/bin/screen -S runbuilds -X quit
> +   WorkingDirectory=/var/www/toaster/poky
> +
>
>
> This is kind of interesting from a developer of Toaster point of view but I
> don't think we should be recommending screen in the production setup.
> Toaster should be logging all the useful output anyway.
>
>
>
> +   [Install]
> +   WantedBy=multi-user.target
>                        </literallayout>
> -                      Here is an example:
> +                      And the runbuilds-service.sh script that you should
> put into /var/www/toaster/poky/bitbake/lib/toaster/ and
> +                      set executable permissions to it.
>                        <literallayout class='monospaced'>
> -   #!/bin/sh
> -   # toaster run builds dispatcher
> -   cd /var/www/toaster/
> -   source ./venv/bin/activate
> -   ./bitbake/lib/toaster/manage.py runbuilds
> +   #!/bin/bash
> +
> +   #export http_proxy=http://proxy.host.com:8080
> +   #export https_proxy=http://proxy.host.com:8080
> +   #export GIT_PROXY_COMMAND=$HOME/bin/gitproxy
> +
> +   cd ~/poky/
> +   source ./oe-init-build-env build
> +   source ../bitbake/bin/toaster $1 noweb
> +   [ "$1" == 'start' ] &amp;&amp; /bin/bash
>                        </literallayout>
> -                      </para></listitem>
> +                  </para></listitem>
> +                  <listitem><para>
> +                      Run the service:
> +                      <literallayout class='monospaced'>
> +   # service runbuilds start
> +                      </literallayout>
> +                      As service is running in a detached screen session
> you can attach to it using this command:
> +                      <literallayout class='monospaced'>
> +   $ sudo su - toaster
> +   $ screen -rS runbuilds
> +                      </literallayout>
> +                      And detach from it again using 'Ctrl-a' followed by
> 'd' key combination.
> +                  </para></listitem>
>                </orderedlist>
>                You can now open up a browser and start using Toaster.
>            </para>
>
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster

[-- Attachment #2: mychecksettings.log --]
[-- Type: application/octet-stream, Size: 5926 bytes --]

2016-07-11 23:54:37,401 DEBUG Fetching branches
2016-07-11 23:54:38,709 DEBUG Fetching layer information
2016-07-11 23:54:44,726 DEBUG Fetching machine information
2016-07-11 23:54:46,783 DEBUG Fetching target information
Traceback (most recent call last):
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.DataError: (1406, "Data too long for column 'license' at row 1")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/var/www/toaster/poky/bitbake/lib/toaster/bldcontrol/management/commands/checksettings.py", line 114, in _verify_be
    call_command("lsupdates")
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/core/management/base.py", line 661, in handle
    return self.handle_noargs(**options)
  File "/var/www/toaster/poky/bitbake/lib/toaster/orm/management/commands/lsupdates.py", line 12, in handle_noargs
    ls.update()
  File "/var/www/toaster/poky/bitbake/lib/toaster/orm/models.py", line 1350, in update
    ro.save()
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/base.py", line 827, in _save_table
    forced_update)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/base.py", line 877, in _do_update
    return filtered._update(values) > 0
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/query.py", line 580, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1062, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/var/www/toaster/.local/lib/python3.5/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1")

 -- Validation: The layers checkout directory must be set.

 -- Validation: The build directory must be set.

Build configuration saved

Importing file: ./meta-poky/conf/toasterconf.json

Layer configuration imported. Updating information from the layer sources, please wait.
You can re-update any time later by running bitbake/lib/toaster/manage.py lsupdates
Failure while trying to import the toaster config file ./meta-poky/conf/toasterconf.json: (1406, "Data too long for column 'license' at row 1")

Re: [PATCH 09/10] toaster-manual: update runbuilds instructions

[Ed Bartosh]

Hi Brian,

Thank you for testing this!

On Mon, Jul 11, 2016 at 05:10:22PM -0700, Brian Avery wrote:
> Hi,
> 
> I was trying out your instructions/production setup on an ubuntu
> 16.04/mysql setup and got the following error:
> $>TOASTER_DIR=`pwd` TOASTER_CONF=./meta-poky/conf/toasterconf.json
> ./bitbake/lib/toaster/manage.py checksettings --traceback 2>&1 | tee
> mychecksettings.log
> 
> resulted in
> 
> django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1")

> I'm attaching the logfile.
> 
> Have any of you seen this before? It kinda stops me cold here. Sadly,
> the error doesn't let me know what table or data was too long.
I've reproduced it. It's failing to load linux-firmware recipe:
https://layers.openembedded.org/layerindex/recipe/59/

With this patch it should skip this recipe with the warning:
https://lists.yoctoproject.org/pipermail/toaster/2016-July/004979.html

> p.s. other issues I've seen so far
> 1) 3.7.2.1
> sudo mkdir /var/www/toaster
> need a : sudo chown toaster /var/www/toaster
Fixed by adding -m option to useradd command:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/python3-documentation&id=f1182c2eee4df48f9dd50c76be540fb0c0578c54

> 2) 3.7.2.4
> SECRET_KEY needs a pointer to the Django documentation so people know
> what it is:
> https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SECRET_KEY
Done. SECRET_KEY, STATIC_ROOT and DATABASE became a links to Django
documentation:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/python3-documentation&id=80f58acb788d552a11d8e1dbcaf233a2ba43b301


> 3) 3.7.2.6
> the TOASTER_DIR=... checksettings line is too long and got truncated
> for me when my pdf was made.  We should probably break it in 2 or use
> exports.
Splitted to 2 lines:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/python3-documentation&id=1bdf15510d96a695e1680db0ee95926e22a4996c

I've updated my branch ed/toaster/python3-documentation with all above
changes. please, review.

Regards,
Ed

> On Fri, Jul 8, 2016 at 6:30 AM, Michael Wood <michael.g.wood@intel.com> wrote:
> > Pasting this in as it's an updated version of this patch:
> >
> >
> > commit 74e368b1855bbf4321f73b09a0f2c973b1e33c61
> > Author: Ed Bartosh <ed.bartosh@linux.intel.com>
> > Date:   Mon Jun 27 17:37:40 2016 +0300
> >
> >     toaster-manual: update runbuilds instructions
> >
> >     Added sample configuration files and instructions for the runbuilds
> >     systemd service. Service runs runbuilds in a screen session for user
> >     convenience. User can attach to the session to see the runbuilds output,
> >     monitor logs and build environment.
> >
> >     Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> >
> > diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> > b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> > index febbd15..26e71c8 100644
> > --- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> > +++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> > @@ -438,22 +438,49 @@
> >                        </literallayout>
> >                        </para></listitem>
> >                    <listitem><para>
> > -                      Install the build runner service.
> > -                      This service needs to be running in order to dispatch
> > -                      builds.
> > -                      Use this command:
> > +                      Prepare systemd service to run toaster builds.
> > +                      Here is a sample configuration file for the service:
> >                        <literallayout class='monospaced'>
> > -   /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
> > +   [Unit]
> > +   Description=Toaster runbuilds
> > +
> > +   [Service]
> > +   Type=forking
> > +   User=toaster
> > +   ExecStart=/usr/bin/screen -d -m -S runbuilds
> > /var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start
> > +   ExecStop=/usr/bin/screen -S runbuilds -X quit
> > +   WorkingDirectory=/var/www/toaster/poky
> > +
> >
> >
> > This is kind of interesting from a developer of Toaster point of view but I
> > don't think we should be recommending screen in the production setup.
> > Toaster should be logging all the useful output anyway.
> >
> >
> >
> > +   [Install]
> > +   WantedBy=multi-user.target
> >                        </literallayout>
> > -                      Here is an example:
> > +                      And the runbuilds-service.sh script that you should
> > put into /var/www/toaster/poky/bitbake/lib/toaster/ and
> > +                      set executable permissions to it.
> >                        <literallayout class='monospaced'>
> > -   #!/bin/sh
> > -   # toaster run builds dispatcher
> > -   cd /var/www/toaster/
> > -   source ./venv/bin/activate
> > -   ./bitbake/lib/toaster/manage.py runbuilds
> > +   #!/bin/bash
> > +
> > +   #export http_proxy=http://proxy.host.com:8080
> > +   #export https_proxy=http://proxy.host.com:8080
> > +   #export GIT_PROXY_COMMAND=$HOME/bin/gitproxy
> > +
> > +   cd ~/poky/
> > +   source ./oe-init-build-env build
> > +   source ../bitbake/bin/toaster $1 noweb
> > +   [ "$1" == 'start' ] &amp;&amp; /bin/bash
> >                        </literallayout>
> > -                      </para></listitem>
> > +                  </para></listitem>
> > +                  <listitem><para>
> > +                      Run the service:
> > +                      <literallayout class='monospaced'>
> > +   # service runbuilds start
> > +                      </literallayout>
> > +                      As service is running in a detached screen session
> > you can attach to it using this command:
> > +                      <literallayout class='monospaced'>
> > +   $ sudo su - toaster
> > +   $ screen -rS runbuilds
> > +                      </literallayout>
> > +                      And detach from it again using 'Ctrl-a' followed by
> > 'd' key combination.
> > +                  </para></listitem>
> >                </orderedlist>
> >                You can now open up a browser and start using Toaster.
> >            </para>
> >
> >
> > --
> > _______________________________________________
> > toaster mailing list
> > toaster@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/toaster



-- 
--
Regards,
Ed

Re: [PATCH] toaster-manual: Update toaster start documentation

[Brian Avery]

[-- Attachment #1: Type: text/plain, Size: 4613 bytes --]

shouldn't we include the more common (for me anyway) case of :
+            For example, the following command sets the host and port to
"0.0.0.0:8400 <http://192.168.0.1:8400/>":

which makes it accessible to other machines on the network? e.g. I tend to
run toaster on my linux box (bigger)  but often use my mac's browser
(nicer)  to interact with it.  If I set the webport to the linux box's ip
address:8400
it accomplishes the same thing but 0.0.0.0 works without my needing to know
what dhcp address I happen to have that day.

-b

On Fri, Jul 8, 2016 at 6:37 AM, Michael Wood <michael.g.wood@intel.com>
wrote:

> - Update starting toaster to source toaster start
> - Add webport address documentation
> - Remove redundant "syncdb" call
> - Make it clearer that collectstatic uses STATIC_ROOT
>
> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
> ---
>  .../toaster-manual-setup-and-use.xml               | 24
> +++++++++++++++++-----
>  1 file changed, 19 insertions(+), 5 deletions(-)
>
> diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> index 66a95ac..1a87b49 100644
> --- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> +++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
> @@ -31,7 +31,7 @@
>              <filename>poky/build</filename>), start Toaster using this
>              command:
>              <literallayout class='monospaced'>
> -     $ source ../bitbake/bin/toaster
> +     $ source toaster start
>              </literallayout>
>              You can now run your builds from the command line, or with
>              Toaster as explained in section
> @@ -56,7 +56,21 @@
>              set a different port.
>              For example, the following command sets the port to "8400":
>              <literallayout class='monospaced'>
> -     $ source ../bitbake/bin/toaster webport=8400
> +     $ source toaster start webport=8400
> +            </literallayout>
> +        </para>
> +    </section>
> +
> +    <section id='setting-a-different-address'>
> +        <title>Setting a Different Address</title>
> +
> +        <para>
> +            By default, Toaster binds to the loop back address i.e.
> localhost.
> +            You can use the <filename>WEBPORT</filename> parameter to
> +            set a different host.
> +            For example, the following command sets the host and port to "
> 192.168.0.1:8400":
> +            <literallayout class='monospaced'>
> +     $ source toaster start webport=192.168.0.1:8400
>              </literallayout>
>          </para>
>      </section>
> @@ -324,7 +338,6 @@
>                        default data, and gather the statically-served
> files:
>                        <literallayout class='monospaced'>
>     $ cd  /var/www/toaster/poky/
> -   $ ./bitbake/lib/toaster/manage.py syncdb
>     $ ./bitbake/lib/toaster/manage.py migrate
>     $ TOASTER_DIR=`pwd` TOASTER_CONF=./meta-poky/conf/toasterconf.json
> ./bitbake/lib/toaster/manage.py checksettings
>     $ ./bitbake/lib/toaster/manage.py collectstatic
> @@ -334,7 +347,7 @@
>                        <para>
>                            For the above set of commands, after moving to
> the
>                            <filename>poky</filename> directory,
> -                          the <filename>syncdb</filename> and
> <filename>migrate</filename>
> +                          the <filename>migrate</filename>
>                            commands ensure the database
>                            schema has had changes propagated correctly
> (i.e.
>                            migrations).
> @@ -375,7 +388,8 @@
>                            Finally, the <filename>collectstatic</filename>
> command
>                            is a Django framework command that collects all
> the
>                            statically served files into a designated
> directory to
> -                          be served up by the Apache web server.
> +                          be served up by the Apache web server as
> defined by
> +                          <filename>STATIC_ROOT</filename>
>                        </para></listitem>
>                    <listitem><para>
>                        Add an Apache configuration file for Toaster to
> your Apache web
> --
> 2.7.4
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>

[-- Attachment #2: Type: text/html, Size: 6570 bytes --]

Re: [PATCH] toaster-manual: Update toaster start documentation

[Scott Rifenbark]

[-- Attachment #1: Type: text/plain, Size: 5127 bytes --]

Brian,

How is this?
http://www.yoctoproject.org/docs/2.2/toaster-manual/toaster-manual.html#setting-a-different-address

Scott

On Wed, Jul 13, 2016 at 9:06 AM, Brian Avery <avery.brian@gmail.com> wrote:

> shouldn't we include the more common (for me anyway) case of :
> +            For example, the following command sets the host and port to
> "0.0.0.0:8400 <http://192.168.0.1:8400/>":
>
> which makes it accessible to other machines on the network? e.g. I tend to
> run toaster on my linux box (bigger)  but often use my mac's browser
> (nicer)  to interact with it.  If I set the webport to the linux box's ip
> address:8400
> it accomplishes the same thing but 0.0.0.0 works without my needing to
> know what dhcp address I happen to have that day.
>
> -b
>
> On Fri, Jul 8, 2016 at 6:37 AM, Michael Wood <michael.g.wood@intel.com>
> wrote:
>
>> - Update starting toaster to source toaster start
>> - Add webport address documentation
>> - Remove redundant "syncdb" call
>> - Make it clearer that collectstatic uses STATIC_ROOT
>>
>> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
>> ---
>>  .../toaster-manual-setup-and-use.xml               | 24
>> +++++++++++++++++-----
>>  1 file changed, 19 insertions(+), 5 deletions(-)
>>
>> diff --git
>> a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>> b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>> index 66a95ac..1a87b49 100644
>> --- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>> +++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>> @@ -31,7 +31,7 @@
>>              <filename>poky/build</filename>), start Toaster using this
>>              command:
>>              <literallayout class='monospaced'>
>> -     $ source ../bitbake/bin/toaster
>> +     $ source toaster start
>>              </literallayout>
>>              You can now run your builds from the command line, or with
>>              Toaster as explained in section
>> @@ -56,7 +56,21 @@
>>              set a different port.
>>              For example, the following command sets the port to "8400":
>>              <literallayout class='monospaced'>
>> -     $ source ../bitbake/bin/toaster webport=8400
>> +     $ source toaster start webport=8400
>> +            </literallayout>
>> +        </para>
>> +    </section>
>> +
>> +    <section id='setting-a-different-address'>
>> +        <title>Setting a Different Address</title>
>> +
>> +        <para>
>> +            By default, Toaster binds to the loop back address i.e.
>> localhost.
>> +            You can use the <filename>WEBPORT</filename> parameter to
>> +            set a different host.
>> +            For example, the following command sets the host and port to
>> "192.168.0.1:8400":
>> +            <literallayout class='monospaced'>
>> +     $ source toaster start webport=192.168.0.1:8400
>>              </literallayout>
>>          </para>
>>      </section>
>> @@ -324,7 +338,6 @@
>>                        default data, and gather the statically-served
>> files:
>>                        <literallayout class='monospaced'>
>>     $ cd  /var/www/toaster/poky/
>> -   $ ./bitbake/lib/toaster/manage.py syncdb
>>     $ ./bitbake/lib/toaster/manage.py migrate
>>     $ TOASTER_DIR=`pwd` TOASTER_CONF=./meta-poky/conf/toasterconf.json
>> ./bitbake/lib/toaster/manage.py checksettings
>>     $ ./bitbake/lib/toaster/manage.py collectstatic
>> @@ -334,7 +347,7 @@
>>                        <para>
>>                            For the above set of commands, after moving to
>> the
>>                            <filename>poky</filename> directory,
>> -                          the <filename>syncdb</filename> and
>> <filename>migrate</filename>
>> +                          the <filename>migrate</filename>
>>                            commands ensure the database
>>                            schema has had changes propagated correctly
>> (i.e.
>>                            migrations).
>> @@ -375,7 +388,8 @@
>>                            Finally, the
>> <filename>collectstatic</filename> command
>>                            is a Django framework command that collects
>> all the
>>                            statically served files into a designated
>> directory to
>> -                          be served up by the Apache web server.
>> +                          be served up by the Apache web server as
>> defined by
>> +                          <filename>STATIC_ROOT</filename>
>>                        </para></listitem>
>>                    <listitem><para>
>>                        Add an Apache configuration file for Toaster to
>> your Apache web
>> --
>> 2.7.4
>>
>> --
>> _______________________________________________
>> toaster mailing list
>> toaster@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/toaster
>>
>
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>
>

[-- Attachment #2: Type: text/html, Size: 7620 bytes --]

Re: [PATCH] toaster-manual: Update toaster start documentation

[Brian Avery]

[-- Attachment #1: Type: text/plain, Size: 6172 bytes --]

Hi,

I'm worried the difference is not coming through.  I'm not sure if it
should be somewhere other than  in Section 3.2  but assuming it were put
there, we could extend 3.2 to say something like:

"""
This means you can access Toaster from a browser running on the same
machine by pointing it to http://localhost:8400.  Toaster would not be
accessible from a different machine on the network using this setup,
however.

To access Toaster from a different machine on the network you need to start
Toaster up as
$> source toaster webport=0.0.0.0:8400 start

Then you could access toaster from a different machine by pointing that
machines browser to <ip address or fqdn of Toaster machine>:8400
""""

Does this help clarify what I'm trying to explain?

Thanks,
-Brian
an intel employee




On Thu, Jul 14, 2016 at 11:14 AM, Scott Rifenbark <srifenbark@gmail.com>
wrote:

> Brian,
>
> How is this?
> http://www.yoctoproject.org/docs/2.2/toaster-manual/toaster-manual.html#setting-a-different-address
>
> Scott
>
> On Wed, Jul 13, 2016 at 9:06 AM, Brian Avery <avery.brian@gmail.com>
> wrote:
>
>> shouldn't we include the more common (for me anyway) case of :
>> +            For example, the following command sets the host and port to
>> "0.0.0.0:8400 <http://192.168.0.1:8400/>":
>>
>> which makes it accessible to other machines on the network? e.g. I tend
>> to run toaster on my linux box (bigger)  but often use my mac's browser
>> (nicer)  to interact with it.  If I set the webport to the linux box's ip
>> address:8400
>> it accomplishes the same thing but 0.0.0.0 works without my needing to
>> know what dhcp address I happen to have that day.
>>
>> -b
>>
>> On Fri, Jul 8, 2016 at 6:37 AM, Michael Wood <michael.g.wood@intel.com>
>> wrote:
>>
>>> - Update starting toaster to source toaster start
>>> - Add webport address documentation
>>> - Remove redundant "syncdb" call
>>> - Make it clearer that collectstatic uses STATIC_ROOT
>>>
>>> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
>>> ---
>>>  .../toaster-manual-setup-and-use.xml               | 24
>>> +++++++++++++++++-----
>>>  1 file changed, 19 insertions(+), 5 deletions(-)
>>>
>>> diff --git
>>> a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>>> b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>>> index 66a95ac..1a87b49 100644
>>> --- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>>> +++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
>>> @@ -31,7 +31,7 @@
>>>              <filename>poky/build</filename>), start Toaster using this
>>>              command:
>>>              <literallayout class='monospaced'>
>>> -     $ source ../bitbake/bin/toaster
>>> +     $ source toaster start
>>>              </literallayout>
>>>              You can now run your builds from the command line, or with
>>>              Toaster as explained in section
>>> @@ -56,7 +56,21 @@
>>>              set a different port.
>>>              For example, the following command sets the port to "8400":
>>>              <literallayout class='monospaced'>
>>> -     $ source ../bitbake/bin/toaster webport=8400
>>> +     $ source toaster start webport=8400
>>> +            </literallayout>
>>> +        </para>
>>> +    </section>
>>> +
>>> +    <section id='setting-a-different-address'>
>>> +        <title>Setting a Different Address</title>
>>> +
>>> +        <para>
>>> +            By default, Toaster binds to the loop back address i.e.
>>> localhost.
>>> +            You can use the <filename>WEBPORT</filename> parameter to
>>> +            set a different host.
>>> +            For example, the following command sets the host and port
>>> to "192.168.0.1:8400":
>>> +            <literallayout class='monospaced'>
>>> +     $ source toaster start webport=192.168.0.1:8400
>>>              </literallayout>
>>>          </para>
>>>      </section>
>>> @@ -324,7 +338,6 @@
>>>                        default data, and gather the statically-served
>>> files:
>>>                        <literallayout class='monospaced'>
>>>     $ cd  /var/www/toaster/poky/
>>> -   $ ./bitbake/lib/toaster/manage.py syncdb
>>>     $ ./bitbake/lib/toaster/manage.py migrate
>>>     $ TOASTER_DIR=`pwd` TOASTER_CONF=./meta-poky/conf/toasterconf.json
>>> ./bitbake/lib/toaster/manage.py checksettings
>>>     $ ./bitbake/lib/toaster/manage.py collectstatic
>>> @@ -334,7 +347,7 @@
>>>                        <para>
>>>                            For the above set of commands, after moving
>>> to the
>>>                            <filename>poky</filename> directory,
>>> -                          the <filename>syncdb</filename> and
>>> <filename>migrate</filename>
>>> +                          the <filename>migrate</filename>
>>>                            commands ensure the database
>>>                            schema has had changes propagated correctly
>>> (i.e.
>>>                            migrations).
>>> @@ -375,7 +388,8 @@
>>>                            Finally, the
>>> <filename>collectstatic</filename> command
>>>                            is a Django framework command that collects
>>> all the
>>>                            statically served files into a designated
>>> directory to
>>> -                          be served up by the Apache web server.
>>> +                          be served up by the Apache web server as
>>> defined by
>>> +                          <filename>STATIC_ROOT</filename>
>>>                        </para></listitem>
>>>                    <listitem><para>
>>>                        Add an Apache configuration file for Toaster to
>>> your Apache web
>>> --
>>> 2.7.4
>>>
>>> --
>>> _______________________________________________
>>> toaster mailing list
>>> toaster@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/toaster
>>>
>>
>>
>> --
>> _______________________________________________
>> toaster mailing list
>> toaster@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/toaster
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 9335 bytes --]