docker notes

To finish the Docker packaging, we should work on the issues below.

1) Conzept environment and build

  • Make sure we have the correct Conzept “settings.conf” (located in the root of the source-tree, the version in “explore2” should be removed) to “/etc/conzept”.
    • Note: The current Docker install copied the template “settings.conf.example” to “/etc/conzept”. (it looked empty to me)
    • Never make the “settings.conf” file readable from the web. Only the “settings.conf.example” should be in the web directory.
  • Install requirements for the Conzept build step:
sudo npm install -g minify minify-json clean-css-cli @swc/cli @swc/core webpack webpack-cli esbuild pm2
  • Run the Conzept build step:
cd $CONZEPT_WEB_DIR/app/explore2/
npm install
npm run build

2) Nginx web server rules

  • Transform the Apache2 URL rewrite rules into Nginx rewrite rules.
    • Note: some rules may not be needed anymore with Nginx. When dropping rules, please note them somewhere (so we can check them later).
  • These apps also use .htaccss rules for pretty-URL support:
    • Dokuwiki (not a priority to fix)

3) Conzept NodeJS services autostart upon boot

The following 3 services should now be build and ready to run (skip the “europeana” app, this app needs to rewritten).

  • allorigins (used for preventing CORS-errors, eg. when fetching an image from Wikipedia)
  • json-proxy (used to hide service API keys in JSON API calls, used by the video app and the Archive Scholar data fetch.)
  • Entitree NodeJS app (Note: Conzept uses a static file tree for this project, no build needed. Building it is quite a hassle. So for now just use the static tree.)

Currently, Conzept uses the “/etc/rc.local” file to startup these background services.

$ cat /etc/rc.local 
#!/bin/sh -e

# read conzept settings
. /etc/conzept/settings.conf

# JSON CORS-proxy (with secret-API-key support, port 50001 on localhost only)
su conzept -s /bin/sh -c '$CONZEPT_SERVICES_DIR/json-proxy/bin/json-proxy -p 50001 -c $CONZEPT_SERVICES_DIR/json-proxy.json &'

# any-file CORS-proxy (port 1458 on localhost only)
su conzept -s /bin/sh -c 'cd "$CONZEPT_SERVICES_DIR/allorigins" && npm start app.js &'

# pm2 services (this service does not start when automatically for some reason, manually it does work)
#pm2 start /var/www/html/conze.pt/services/entitree/start.sh

exit 0;

We also need this cronjob line:

# fetch Conzept cover-data (JSON files) for the previous month (we wait until the second day of the month, so the Wikipedia popularity stats are available)
0 0 2 * * su - www-data -s /bin/sh -c . /etc/conzept/settings.conf ; cd $CONZEPT_WEB_DIR$CONZEPT_BASE_DIR/app/explore2/tools/ && sh ./get_previous_month_covers.sh

4) configure the NodeJS services for use with Nginx

Use the correct reverse-proxy settings (see the old Apache2 settings below):

     <Location /app/proxy>
        # command: /home/user/bin/conzept-api-proxy.sh 
        ProxyPass http://127.0.0.1:50001
        ProxyPassReverse http://127.0.0.1:50001
     </Location>

     <Location /app/cors>
        # command: npm start /home/user/dev/allOrigins/app.js
        ProxyPass http://127.0.0.1:1458
        ProxyPassReverse http://127.0.0.1:1458
     </Location>
     
     <Location /app/tree>
        ProxyPass http://127.0.0.1:3000
        ProxyPassReverse http://127.0.0.1:3000
     </Location>     

5) Docker workflows

  • Describe (and I'll document) a good workflow to develop Conzept, update the site and update the Git repo.
  • Describe (and I'll document) how we can use the Docker package to allow for localhost development.
  • How to best maintain the Conzept Dokuwiki documentation (split the Conzept guide out of the Docker package? Or generate a static-site and include that?)

final tests

Tests that Conzept should work as normal:

  • Do another Docker install from a fresh VPS setup → Check if everything is running fine. (I'll run this test)
  • Test development workflow (currently: I develop directly on production and merge changes with Github when the code changes are good)