(2012-10-16) Setting Up Hosting For Simplest Thing

Pretty much done with v1 of Simplest Thing, time to put up on a Hosted Server.

PaaS vendors are too expensive when you're a Micropreneur putting up a paid (thus low-load) service.

Going with Linode, largely because friend offered to help there. Otherwise probably would have gone with Web Faction. Just $20/mo.

(Have built with WebPy - WebpyForSimplest Thing.)

Looks like will end up running with NginX, Postgre[[SQL]], and WSGI (uWSGI - UWsgi).

note that page is 18mo old, so will pick newer versions of things to install.
- Ubuntu 11.10
- uwsgi-1.2.6
- nginx-1.3.5

When get to point of launching NginX, get Starting nginx: nginx: [emerg] getpwnam("nginx") failed grrr how much do I hate it when recipes don't work.

had to play some games with setting owner in nginx.conf and doing chown/chmod stuff. Done Oct17.

Config NginX virtual server.

mostly taking background info from here
- except looks like files will be served from /srv/www/ not /var/www/
just add a server block inside nginx.conf instead of playing with /sites-available/
- and making the /static directory a sub of /app/ where my code will be, since that's where WebPy tends to put things.
/etc/init.d/nginx restart server - comes up after I do /mkdir/ for the /logs/
but hitting the IP# doesn't give me anything. Is that because I need to make requests by the hostnames used in the server block?
- made local hosts entry to point to server by name. Still not connecting.
- played chown/chmod games in /srv/www/ - no change
- noting that I've been playing with 2 different nginx.conf files - one in /opt/nginx/ and the other in /opt/nginx-1.3.5/ - and today's edits were only in the former.
- find /opt/nginx/logs/error.log saying 2012/10/17 09:10:04 [emerg] 21887#0: open() "/srv/www/simplest-thing.com/logs/access.log" failed (2: No such file or directory). So copy the 2 log files from /opt... to /srv... (at 10:18am). Restart server again. Same result. No additions to either set of log files.
- hmm, could problem be that I added that new server block to nginx.conf but left in already-existing block that also uses port-80 (for localhost)?
  - tweak that original server block to listen on 81. Restart. Still won't connect, still nothing added to logs.
  - do chown/chmod on logs in both directories. Restart. Still won't connect, still nothing added to logs.
- grr, should have checked that server was actually running before I started this step.
- my friend isn't available, what can I try on my own?
  - rip out the Server Block I added? No, actually start back from default.
  - play with the fact that there's 2 config files - maybe rename 1 to get it ignored?
    - renamed /opt/nginx-1.3.5/conf/nginx.conf to nginx.conf.seitz. Restart, no change.
  - in /opt/nginx/conf rename my latest nginx.conf to nginx.conf.seitz and then copy nginx.conf.default to nginx.conf
  - do chown/chmod on that file again. No change.
  - chang user line inside nginx.conf - try user name my friend had said, also tried nginx. Neither worked. (And still nothing in any nginx logs, nor anything useful in /var/log/.
- my friend finds simple fix - just change line in nginx.conf that says to listen on localhost to listen on the specific IP (Oct22)
  - I confirm that http://173.255.235.226/ and http://www.simplest-thing.com/ (because of my hacked hosts file) both give me the base nginx page.
  - also confirm that hits are going to /opt/nginx/logs/ as per nginx.conf
I edit nginx.conf to add server block per instruction page. Restart. Now get 502 Bad Gateway response.
- the /opt/nginx/logs/access.log was just updated with line 76.245.240.208 - - [22/Oct/2012:16:23:43 -0500] "-" 400 0 "-" "-"
- the /srv..../access.log was updated with line {{{ 76.245.240.208 - - [22/Oct/2012:16:23:33 -0500] "GET / HTTP/1.1" 502 574 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) Apple Web Kit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.26 Safari/537.11" 76.245.240.208 - - [22/Oct/2012:16:23:33 -0500] "GET /favicon.ico HTTP/1.1" 502 574 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) Apple Web Kit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.26 Safari/537.11" }}}
- the /srv/.../error.log was updated with {{{ 2012/10/22 16:23:33 [error] 25135#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 76.245.240.208, server: www.simplest-thing.com, request: "GET / HTTP/1.1", upstream: "uwsgi://127.0.0.1:9001", host: "www.simplest-thing.com" 2012/10/22 16:23:33 [error] 25135#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 76.245.240.208, server: www.simplest-thing.com, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://127.0.0.1:9001", host: "www.simplest-thing.com" }}}
- edit nginx.conf file again to change the original server blog to listen on 81 instead of 80. Restart. No change.
- do mkdir /srv/www/simplest-thing.com/app/static/ then create index.html inside it. Restart. Try to hit http://www.simplest-thing.com/static/index.html get 404 Not Found error. Log looks similar, though notice some lines like 2012/10/22 16:41:49 [error] 25233#0: *1 open() "/srv/www/simplest-thing.com/app/static/static" failed (2: No such file or directory), client: 76.245.240.208, server: www.simplest-thing.com, request: "GET /static HTTP/1.1", host: "www.simplest-thing.com" mean I have static being mapped twice, or I'm not supposed to include static in the URL, or something.
- I also try hitting http://www.simplest-thing.com/wsgi_configuration_module.py for kicks, but get 502. Should a request for / be automatically calling that to give me the Hello World response?
- kicking it back to my friend.... who says...
- derp you need to launch a uwsgi daemon that runs on port 9001. See her comments below. Also see this page that I just found.

Install Postgre[[SQL]] - apt-get install postgresql python-psycopg2

Install WebPy - apt-get install python-webpy

Use scp to copy all my app stuff from my Lap Top up to server - in /srv/.../app/.

includes .sql dump

Next: set up Postgre[[SQL]] database and user so can import from dump.

createuser webpy
createdb mf
hrm, I should probably switch over on my Lap Top, to avoid having to do this repeatedly. (And to avoid environment-specific code changes.)
ah, can't use pgdump to import data until convert that dump file from SQLite style.
- start with these notes to make edits to dump file
  - make ID fields serial primary key not just serial
- delete sqlite_sequence table and data.
- re-arrange tables so that "base" tables come before detail-tables that reference them
- had a table with lots of records having an empty price value that was set like empty-string instead of null or 0. Changed them all to 0.
- had a field that was supposed to be just varchar(8) but had many values that were longer than that! Changed field spec.
- had a text field set to varchar(1024) but max length of data I was importing was 1070. Changed all such fields to type text.
- success!
hrm, psql stores each serial/sequence in its own table. So I have a number of tables created, but each has all its values set to 1, like {{{ sequence_name | last_value | start_value | increment_by | max_value |min_value | cache_value | log_cnt | is_cycled | is_called ---------------+------------+-------------+--------------+---------------------+----------+-------------+---------+-----------+----------- notes_id_seq | 1 | 1 | 1 | 9223372036854775807 | 1 | 1 | 0 | f | f }}}
- list all the sequences with select relname from pg_class where relkind = 'S'; (Oct14)
- for each case, do SELECT setval('your_table_id_seq', (SELECT MAX(id) FROM your_table));

Edit my WebPy code.py for new db settings. (Will have to make config system later.) Use user='webpy' with no password.

test this user theory by doing commandline psql user=webpy mf
- get psql: FATAL: Peer authentication failed for user "webpy"
- re-created user webpy with a password
- edit /etc/postgresql/9.1/main/pg_hba.conf for its local connection use md5
- restart psql sudo service postgresql restart
- test connect with command line - get prompted for password
- change WebPy code.py to include password param.

Get uwsgi pointing to my code.py

edit /etc/default/uwsgi (though suspect that doesn't matter)
edit start_uwsgi_app.sh created by my friend
use pgrep uwsgi to find the running uwsgi processes, then kill each
do bash start_uwsgi_app.sh to start new uwsgi
hit url with browser, get uWSGI Error | Python application not found
this doesn't surprise me, as I had already seen some instructions for using web.py with uwsgi that require more tweaks to code. But figured I try anyway.
add application = app.wsgifunc() to the very end of the code.py (even outside the __name__ == __main__ test)
add sys.path.append('/srv/www/simplest-thing.com/app') up at the top of code.py
kill/restart uwsgi
hit URL again
- redirects me to my static splash page as it's supposed to! (since I'm not logged in)
- but not using the CSS or images it's supposed to
- I view-source and click one of the asset links - get 403/Forbidden
- play with chmod in the /static/splash/assets/ directory, now all look identical to files inside /static/assets/img/ - like -rwxr-xr-x 1 root dev 264 2012-10-22 21:02 icon-twitter.png - but the latter I can get in browser, and the former I can't!
- ah, the issue was that the latter /assets/ container didn't have chmod a+rx
- now when I hit the splash page I get the pretty version!
now request page that's public (no login needed) but dynamic - http://www.simplest-thing.com/resources_public
- get permission denied for relation resources error. Suspect the user webpy doesn't have the appropriate rights inside psql. But off to bed....
- yes, went into psql command-line with user webpy - could get in (as I did yesterday), but when I tried to actually do a select I got the same error message.
- went back in as use postgres, did GRANT ALL PRIVILEGES ON DATABASE mf to webpy, quit, went back in as webpy, hit same wall.
- list permissions: {{{ SELECT datname as "Relation", datacl as "Access permissions" FROM pg_database WHERE datname = 'mf'; Relation | Access permissions
  ----------+--------------------------------------------------------- mf | {=Tc/postgres,postgres=CTc/postgres,webpy=CTc/postgres} }}}
  - hrm what does that mean? Hm doesn't look like ALL PRIVILEGES were granted.
  - ah, that command doesn't do what it sounds like.
  - so did GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO webpy;
  - now it works! (in command-line)
- now hit that resources_public URL again - it works!

Try to log in to web app.

Whoops No module named cryptacular.bcrypt. So do easy_install cryptacular then kill/restart uwsgi.
Try again: permission denied for sequence action_log_id_seq
I wonder if it makes more sense to make webpy the owner of the whole database?
Nah, just did Grant usage, select on sequence action_log_id_seq to webpy; for each sequence. (Oct26)
also had to easy_install markdown
discovered I was mis-using db.insert() in terms of returning the created ID value: you have to pass the db-sequence name associated with the ID.
also discovered I couldn't have a SQL query in code which used a literal in double-quotes, as psql doesn't like that.

Edited: 2013-02-17 00:00:00 | Tweet this! | Search Twitter for discussion

Bill Seitz