docker notes

To finish the Docker packaging, we should work on the issues below.

• Make sure we have the correct Conzept “settings.conf” (located in the root of the source-tree, the version in “explore2” should be removed) to “/etc/conzept”.
  • Note: The current Docker install copied the template “settings.conf.example” to “/etc/conzept”. (it looked empty to me)
  • Never make the “settings.conf” file readable from the web. Only the “settings.conf.example” should be in the web directory.

• Install requirements for the Conzept build step:

sudo npm install -g minify minify-json clean-css-cli @swc/cli @swc/core webpack webpack-cli esbuild pm2

• Run the Conzept build step:

cd $CONZEPT_WEB_DIR/app/explore2/

npm install

npm run build

• Transform the Apache2 URL rewrite rules into Nginx rewrite rules.
  • Note: some rules may not be needed anymore with Nginx. When dropping rules, please note them somewhere (so we can check them later).

• These apps also use .htaccss rules for pretty-URL support:
  • Dokuwiki (not a priority to fix)

The following 3 services should now be build and ready to run (skip the “europeana” app, this app needs to rewritten).

• allorigins (used for preventing CORS-errors, eg. when fetching an image from Wikipedia)
• json-proxy (used to hide service API keys in JSON API calls, used by the video app and the Archive Scholar data fetch.)
• Entitree NodeJS app (Note: Conzept uses a static file tree for this project, no build needed. Building it is quite a hassle. So for now just use the static tree.)

Currently, Conzept uses the “/etc/rc.local” file to startup these background services.

$ cat /etc/rc.local 
#!/bin/sh -e

# read conzept settings
. /etc/conzept/settings.conf

# JSON CORS-proxy (with secret-API-key support, port 50001 on localhost only)
su conzept -s /bin/sh -c '$CONZEPT_SERVICES_DIR/json-proxy/bin/json-proxy -p 50001 -c $CONZEPT_SERVICES_DIR/json-proxy.json &'

# any-file CORS-proxy (port 1458 on localhost only)
su conzept -s /bin/sh -c 'cd "$CONZEPT_SERVICES_DIR/allorigins" && npm start app.js &'

# pm2 services (this service does not start when automatically for some reason, manually it does work)
#pm2 start /var/www/html/conze.pt/services/entitree/start.sh

exit 0;

We also need this cronjob line:

# fetch Conzept cover-data (JSON files) for the previous month (we wait until the second day of the month, so the Wikipedia popularity stats are available)
0 0 2 * * su - www-data -s /bin/sh -c . /etc/conzept/settings.conf ; cd $CONZEPT_WEB_DIR$CONZEPT_BASE_DIR/app/explore2/tools/ && sh ./get_previous_month_covers.sh

Use the correct reverse-proxy settings (see the old Apache2 settings below):

     <Location /app/proxy>
        # command: /home/user/bin/conzept-api-proxy.sh 
        ProxyPass http://127.0.0.1:50001
        ProxyPassReverse http://127.0.0.1:50001
     </Location>

     <Location /app/cors>
        # command: npm start /home/user/dev/allOrigins/app.js
        ProxyPass http://127.0.0.1:1458
        ProxyPassReverse http://127.0.0.1:1458
     </Location>
     
     <Location /app/tree>
        ProxyPass http://127.0.0.1:3000
        ProxyPassReverse http://127.0.0.1:3000
     </Location>     

• Describe (and I'll document) a good workflow to develop Conzept, update the site and update the Git repo.

• Describe (and I'll document) how we can use the Docker package to allow for localhost development.

• How to best maintain the Conzept Dokuwiki documentation (split the Conzept guide out of the Docker package? Or generate a static-site and include that?)

Tests that Conzept should work as normal:

• TODO: some specific URLs to test:
  • basic URL
  • article view
  • images view
  • audio-app view (needs CORS-proxy running)
  • enitree view (needs entitree NodeJS-app running)

• Do another Docker install from a fresh VPS setup → Check if everything is running fine. (I'll run this test)

• Test development workflow (currently: I develop directly on production and merge changes with Github when the code changes are good)