WebSeitz/wiki

(2022-08-25) Move webseitz to DigitalOcean

I moved FluxGarden to DigitalOcean, but nothing else at LiNode. So webseitz is still running from there...

Where do I want to end up?

Later

  • then generate WikiGraph footers directly in FluxGarden code, dumping remove WikiGraph call
  • (ideally/later) have webseitz start using ssl, leave non-ssl running (pick SSL to be canonical, and let people's browsers nudge them to it)
  • then bring WikiGraph code over (have to move from web.py to flask but that should be easy) - support with SSL and non-SSL depending on the preference of the site who's embedding it

Where are things now?

  • whereis nginx -> nginx: /usr/sbin/nginx /usr/lib/nginx /etc/nginx /usr/share/nginx /usr/share/man/man8/nginx.8.gz
  • nginx -v -> nginx version: nginx/1.18.0 (Ubuntu)
  • /etc/nginx/nginx.conf is fairly generic, meat is in /etc/nginx/sites-enabled/flux.garden.conf
    • some interesting ranting against sites-enabled, but I'm not going to change it (I appear to have the same flux.garden.conf file in both sites-enabled and sites-available, rather than a symlink. Also, I've never used the scripts like nginx_ensite
  • I have nothing inside /etc/nginx/conf.d
  • other key config files seems to be /etc/systemd/system/wikiweb.service and /home/bill/wikiweb/wikiweb_uwsgi.ini

So the big/first question for me is: do I even need a separate service running to handle the new domain, if it's the same code being used?

  • friend recommends: one nginx instance, two redirect rules to two different proxies, two different flask daemons, two different systemd scripts

Feb26'2023

  • the existing service is called wikiweb, I'm going to call the new one fluxent
  • :/etc/systemd/system$ sudo cp wikiweb.service fluxent.service then edit
  • ~/wikiweb$ cp wikiweb_uwsgi.ini fluxent_uwsgi.ini then edit
  • there's a ~/uwsgi_sockets/wikiweb.socket which seems to symlink to itself? Do I manually create this, or does it get created automatically if I define that elsewhere (I think in the nginx config)? We'll move on and see....
  • /etc/nginx/sites-available$ cp flux.garden.conf fluxent.com.conf then edit (not supporting SSL this round) (and copy file to ../sites-enabled/
  • sudo systemctl restart fluxent....
  • ps awux | grep fluxent → see bunch of processes
  • need to edit my laptop hosts to point there to try hitting it → nope, redirects to flux.garden
  • probably need to restart nginx → service nginx restart → wants a password
  • sudo nginx -s reload → nope redirects to flux.garden

Feb27

  • set debug-logging for nginx - do I have SSL turned on for fluxent.com? Doesn't look like it...
  • ah, the redirect to flux.garden is happening inside the Flask code! Yep, confirmed in the flask/uwsgi log.
  • ugh, SERVER_NAME is hard-coded in .env - will there be an issue if I make it more dynamic? Do I need a separate flask install for each domain?
  • also note: running
Python 3.8.5
Flask 1.1.2
Werkzeug 1.0.1

Apr23

flask --version
Python 3.8.5
Flask 1.1.2
Werkzeug 1.0.1

Apr26

  • I realize I've never had subdomain_matching=True set, so I add it
  • decorator line is now like @app.route('/<path>/<page_url>', subdomain='<subdomain>', host="flux2.garden:5000")
  • → TypeError: page() missing 1 required positional argument: 'subdomain' so I guess the subdomain parsing isn't working
  • I asked ChatGPT about host_matching, and it gave me some examples!
  • @app.route('/<path>/<page_url>', subdomain='*', host='flux2.garden:5000') → now getting 404 again!
  • @app.route('/<path>/<page_url>', subdomain='webseitz', host='flux2.garden:5000') → also 404!
  • I have a logger-line right inside the page() function, and it doesn't get called, so the issue is in the route matching, not something inside the function.
  • ChatGPT also suggested adding host_matching to the route decorator → @app.route('/<path>/<page_url>', subdomain='webseitz', host='flux2.garden:5000', host_matching=True) → nope TypeError: __init__() got an unexpected keyword argument 'host_matching'
  • try getting rid of subdomain as separate argument
@app.route('/<path>/<page_url>', host='http://webseitz.flux2.garden:5000')
def page(page_url, path, host):
  • → nope 404

May01: decide i don't trust my local repo

  • download from gitlab
  • copy .env and .flaskenv from local-old folder into fresh folder; flask run works
  • hit http://webseitz.flux2.garden:5000/ → KeyError: SERVER_NAME (which makes sense, since I deleted it from .env as part of host_matching)
  • same error → oh yeah have to remove SERVER_NAME line from config.py, too
  • now I get the 404 again

Aug27 reconsidering my options for moving forward

  • Fresh venv, fresh flask, copy app, launch, make multi host
  • Upgrade DO. Install separate code copy, live with it.
  • yep, decide the latter, it's making more sense to me to have 2 similar-but-independent code bases, esp since I want to noodle on some new ideas... but I will aim to run them both from the same db (not just server, but specific db)
  • make new fluxent directory. git clone into it (using https) -> /fluxent/wikiweb/
  • that doesn't copy .env file, so do it by hand
    • edit SERVER_NAME and SERVER_PROTOCOL (just http not https)
  • mv fluxent_uwsgi.ini ~/fluxent/wikiweb/.
    • edit paths in file
  • edit /etc/nginx/sites-available/fluxent.com.conf (and copy to sites-enabled)

Aug30

  • do sudo nginx -s reload -> nginx: [warn] conflicting server name "fluxent.com" on 0.0.0.0:80, ignored
  • sniffing around, notice I have both fluxent_uwsgi.ini and wikiweb_uwsgi.ini in ~/fluxent/wikiweb (and the latter file is also over in its proper directly). So rm wikiweb_uwsgi.ini in the fluxent tree. Nope, doesn't change anything
  • maybe I need more than reload, do sudo systemctl restart nginx - no change
  • if I do ps awux | grep fluxent I only see 1 process, whereas if I do it for wikiweb I see a whole bunch.
  • look inside /var/log/nginx/error.log notice connect() to unix:///home/bill/uwsgi_sockets/fluxent.socket failed (2: No such file or directory)
  • go into ~/uwsgi_sockets find wikiweb.socket, no fluxent.socket
  • /etc/nginx/sites-available/fluxent.com.conf contains uwsgi_pass unix:///home/bill/uwsgi_sockets/fluxent.socket;
  • maybe chown issues? (though everything is consistent across the old and new apps)
    • the app-code folders all have bill:bill
    • ~/uwsgi_sockets are bill:bill
    • /var/log/nginx/* are all www-data:adm
    • /var/log/uwsgi/* are all bill:bill (and they're all empty)
    • /etc/nginx/* are all root:root

Sept04

  • derp I haven't been manually killing my uwsgi services, so I think restarting nginx tries to trigger things running alongside the old processes...
  • sudo pkill -f uwsgi -9 -> bunch of uwsgi processes immediately respawned
  • (noticing that up at Feb26 I had a "bunch" of fluxent processes running but now doing ps awux | grep fluxent just gives me one
  • sudo systemctl restart fluxent -> Job for fluxent.service failed because the control process exited with error code. See "systemctl status fluxent.service" and "journalctl -xe" for details.
  • journalctl -xe -> realpath() of /home/bill/wikiweb/fluxent_uwsgi.ini failed: No such file or directory [core/utils.c line 3662]... fluxent.service: Executing: /bin/sh -c 'exec /usr/local/bin/uwsgi --ini /home/bill/wikiweb/fluxent_uwsgi.ini'
  • ah, /etc/systemd/system/fluxent.service needs to be edited
  • edit, restart -> same error msg
  • sudo grep -Rn '/' -e '/bill/wikiweb/fluxent'
  • derp had the wrong path-order inside fluxent.service
  • kill/restart -> same error
    • except journalctl -xe doesn't give me anything fresh
  • systemctl status fluxent.service ->
fluxent.service - fluxent uWSGI Dev
     Loaded: loaded (/etc/systemd/system/fluxent.service; disabled; vendor preset: enabled)
     Active: failed (Result: resources) since Mon 2023-09-04 18:32:54 UTC; 1min 26s ago
  • found post that suggested checking syslog (because should give detail behind the Result: resources bit)
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Installed new job fluxent.service/restart as 1004362
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Enqueued job fluxent.service/restart as 1004362
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Scheduled restart job, restart counter is at 5.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202270 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202271 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=JobNew cookie=202272 reply_cookie=0 signature=uos error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202273 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202274 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Changed auto-restart -> dead
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Job 1004362 fluxent.service/restart finished, result=done
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Stopped fluxent uWSGI Dev.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Converting job fluxent.service/restart -> fluxent.service/start
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Start request repeated too quickly.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Failed with result 'resources'.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Service restart not allowed.
  • tried manual command-line /bin/sh -c 'exec /usr/local/bin/uwsgi --ini /home/bill/fluxent/wikiweb/fluxent_uwsgi.ini' -> no error, and bunch of fluxent processes running. And can his the URL that was failing (with 502) before!
    • and now getting non-empty /var/log/uwsgi/wikiweb.log and fluxent.log
  • if I ctl-c the uwsgi window, all the fluxent processes go away, and I start getting the 502 again.
    • and FYI the tail of fluxent.log is
SIGINT/SIGQUIT received...killing workers...
worker 1 buried after 1 seconds
worker 2 buried after 1 seconds
worker 3 buried after 1 seconds
worker 4 buried after 1 seconds
worker 5 buried after 1 seconds
worker 6 buried after 1 seconds
worker 7 buried after 1 seconds
worker 8 buried after 1 seconds
goodbye to uWSGI.
VACUUM: pidfile removed.
VACUUM: unix socket /home/bill/uwsgi_sockets/fluxent.socket removed.

Oct08'2023 - friend is creating new infrastructure for me, involving Docker in some way... so while waiting I'm going to focus on getting my local code working again, ripping out any of the multi-domain stuff... I'm going to focus on my wikiweb-main folder, checking the config bits from my wikiweb_20210425 snapshot

  • ah, it was the __init__.py, ripping the new junk of out app = Flask(__name__)

May26'2024: friend has had issues, want to try again alone.

  • also (2024-04-28) Updating my MacBookPro Python
  • Current thinking: try to get the simplest possible hello-world Flask app running in 2nd domain - even just as server-local app not hittable from my computer?
  • at ~ I have directories electqq fluxent hold snap uwsgi_sockets w2 wikiweb wikiweb.gz wikiweb.sql
    • wikiweb is FluxGarden, fluxent is for me
  • cd into fluxent/wikiweb and do flask run ->
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
[2024-05-26 21:07:31,551] INFO in __init__: wikiflux startup
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
/home/bill/.local/lib/python3.8/site-packages/flask/app.py:2171: UserWarning: Current server name '127.0.0.1:5000' doesn't match configured server name 'fluxent.com'
  return self.url_map.bind_to_environ(
2024-05-26 21:09:47,595 INFO sqlalchemy.engine.Engine select version()
  • and in separate window: curl http://127.0.0.1:5000/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
  • so obviously just launching the server launches a bunch of pieces that are not "simplest thing"
  • so I think I want to start a fresh directory, with simplest app, then move toward getting it working with nginx and uwsgi...
  • (also, as I look at the linked article, I suspect some hack just took a generic Flask intro and did some mindless search/replace, bits of this don't really make sense at DigitalOcean)

Jul14'2024

  • going to trust that linked doc above; also going to assign a 3rd domain that I'm not using to point at.
  • already have directory ~/flaskHelloWorld, cd into it
    • python -> 3.8.10
    • python3 -m venv .venv -> The virtual environment was not created successfully because ensurepip is not available....
    • sudo apt install python3.8-venv
    • python3 -m venv .venv
    • source .venv/bin/activate
    • flask run - yes, is already installed, now running on http://127.0.0.1:5000/
    • open other terminal window into digital ocean, curl http://127.0.0.1:5000/ -> Hello world, all good
    • try to hit http://billseitz.com:5000/ - get Couldn't connect to server so going to....
  • Move on to uWSGI and nginx
    • early bits are redundant, skip down to...
    • If you followed the initial server setup guide, you should have a UFW firewall enabled. To test the application, you need to allow access to port 5000:... (ah maybe this is why my earlier attempt didn't connect)
    • sudo ufw allow 5000 -> ok
    • hit http://billseitz.com:5000/ nope still no connection; nor with the IP#
    • so I feel like something else is going on, not relevant to uwsgi/nginx - maybe a security thing?
  • I see where I set up ufw at Linode, but I don't have the equivalent logged here. But maybe my friend did it? How can I tell what's set up here?

Edited:    |       |    Search Twitter for discussion

Made with flux.garden