(2020-10-12) Building user management in WikiFlux

Building user management in WikiFlux so it will be multi-user.

What kinda flow do I want?

  • self-service to create
    • create "user"
    • then pay, which then activates user which allows actual creation/editing (and links/redirects into the space)
  • login: hit link on any page, submit, get redirected back to self? Or root, I'm lazy.
  • view-vs-edit, privacy, etc.
    • can view public page anonymously, have login link in menu as today.
    • when viewing private page, or rendering Edit link for a view-public-page, have to check that the user is the owner of that space!


  • pip install Flask-User → success
  • going to have to manually alter the users table schema.
    • am I going to use the role feature? Or do something nastier to limit admin rights to my 1 user?
    • probably keep it simple to start
    • for users table, make a SQL script I can run later in prod. Actually 2 bits, because have to modify the existing record before adding constraints. They're saved in file, but I ran them in command-line.
  • edit models.py
  • launch app →
class User(Base, UserMixin):
 NameError: name 'UserMixin' is not defined
  • made some tweaks, changed to
    'No Flask-SQLAlchemy, Flask-MongoEngine or Flask-Flywheel installed and no Pynamo Model in use.'\
flask_user.ConfigError: No Flask-SQLAlchemy, Flask-MongoEngine or Flask-Flywheel installed and no Pynamo Model in use. You must install one of these Flask extensions.
  • so it's not seeing the db connection. I wonder if I need to change my other code to use db = SQLAlchemy(app) but that probably requires a bunch of other changes. Maybe need to find some more example Flask apps, conform my code?
  • Oct18 hacked the flask-user db_manager code to assume it's sqlalchemy


  • added email-config values - use bill@simplest-thing.com
  • it didn't like my secret_key, I had to muck around to satisfy it
  • now app is running!
  • paste in code for /members route
  • hit /members get sign-in page; enter info I had pushed into db before → get error, derp I added column email_confirmed should be email_confirmed_at
  • ALTER TABLE to fix column; hit again → UnknownHashError: hash could not be identified hmm not lots of help there
  • try registering new account bill@simplest-thing.comAttributeError: 'scoped_session' object has no attribute 'session'
  • hack couple lines in flask-user to turn db.session. into db.
  • submit new-user again → fails because flask-user isn't creating id - null value in column "id" of relation "users" violates not-null constraint
  • while my code refers to field as auto-increment, nothing in the schema creates that.
    • hmm should I use field type serial? How convert to that? Nope, user more SQL-compliant identity/generated
    • once I settle on type, I can probably drop the existing id field then add the new type
    • decide on this


  • fix schema ^^
  • submit again - realize form is missing lots of fields I made required. Do I really want the user to have to enter them all to create an account? Probably....
  • derp they have a built-in template, I'll try the standard syntax to use that template!


  • the built-in template only uses the couple fields, too. Gonna make local copy and add my fields. No still seems to be using their form.
  • change my route to /register, now get an error about form object for hidden_tag - research says that's for WTForms which is for CSRF protection, so I should be doing that anyway.
  • step back a sec - confirm app is working fine still with old hard-coded admin login, so I have a safe base to work form. So going to put in WTF/WTForms first. Get that working with old-model, then move forward.
  • pip install flask-wtf → I think it was automatically installed by flask-user
  • put in some code, but template is still unchanged.... it works
  • add in form.validate_on_submit() part - dies hard


  • changed it to just use validate yeah it fails in there
  • ah probably forms class has to have fields matching what's in my manual form derp
  • still failing
  • derp had to put in {{ form.csrf_token }} in my form, boom!
  • (also confirmed making-new-page works)
  • next - back to Flask-User


  • got to next error, I need to define the form class
    • I can [extend}(https://flask-user.readthedocs.io/en/latest/customizing_forms.html#customizingformclasses) the RegisterForm class (to add fields), or copy/paste/fork. I'm going to try extending.
  • tried forking, couldn't figure out how to import validators to call, etc. so went back to extending
  • finally got form to load
  • but have no method to submit yet


  • before I add any user records, I need to get that serial/generated thing working.
  • ok it's done (did I just do it, or the other day?) - did pgdump of schema
  • add more fields to schema, model, form, field
  • next: POST logic
  • unclear from examples how much is manual to get the form data, and then save to db
  • try to see if validate_and_submit() does the job, appear not.
  • ah, key method is populate_obj()
  • but fail: was a class (models.User) supplied where an instance was required? - just needed parens
  • now running but not inserting


  • ah print user says the populate_obj() call isn't actually doing anything....
  • oh derp actual problem was that I wasn't calling commit!
  • now have issue with username is empty, despite form... (which then triggers uniqueness issue)


  • try ripping out bits of form logic, still no change. Other fields (both built-in and added-by-me) are handled fine.


  • realize I wasn't following the right approach to customizing form templates. Move my custom form, change code to map to it. Form loads, but now submit passes all null data.
  • derp some of the doc pages I've been using are for 0.6 instead of 1.0!
    • different code change
    • nope all still null!
  • uncommented form.populate_obj(user) - now getting fields except for username again
  • trying adding username to MyRegisterForm() - didn't help, took it back out
  • asking for help in github


  • wondering if I should step back and align my code's structure with a more "standard" Flask model. But what the heck is that?
  • Do I have to adopt Blueprints?

Plan: I'm going to suck it up and upgrade to current Flask, then restructure in line with this


  • pip install --upgrade Flask no complaints. Runs fine!
  • tutorial puts a venv inside the app's directory, then an app in there. I'm not putting the venv down at that level. But I will make an app subdirectory of my app's directory
  • make wikiweb.py above app with just from app import app
  • make __init__.py inside app with the app = Flask(__name__) and similar bits
  • rename old core file wikiweb.py to routes.py. Rip out the main caller bits, swap in from app import app for past app-import bits.
  • set environment variable from command-line
  • flask runflask_user.ConfigError: Config setting USER_EMAIL_SENDER_EMAIL is missing. even though it's there in the routes.py file. Should probably move some config stuff around.
  • so jump ahead to section 3 - make config.py above /app/
  • flask run → launches! Hit page - works!
  • go back to section1, do pip install python-dotenv and make .flaskenv file. Still runs
  • section2: I already have a templates subdirectory with templates, and they are working since my pages are loading.
  • section3: I have WTForms, but maybe now's a good time to play with that directly for a non-user/login form...
  • create new page, leave body empty, submit → "Internal Server Error", see my form.validate() was false, with form.errors = {'body': [u'This field is required.']}


  • ah, I see, they use validate_on_submit() so that if validation fails they can treat it like a GET and deliver the form. Would need nice error handling, but that kinda makes sense....
  • tweak to use that → happy paths work; missing-field silent (to user) but doesn't explode
  • I already had flash-messages in my layout template, now call flash if validate fail


  • section iv - databases
  • switch to db instead of db_session, stop importing the database.py file that calls scoped_session, move config bits into config.py
  • launch → get error on user_manager = CustomUserManager(app, db, User) because db is not defined - suspect this is part of the kludging I did to get flask-user working, so will go back to their raw instructions....
  • follow their starter-app organization - move UserManager bits into __init__.py
  • now launches. Hit page → db not defined - still need a from app import db in routes.py
  • then need to comment out the db shutdown/remove bits
  • now working - to view page
  • edit → 'SQLAlchemy' object has no attribute 'commit'
  • change all the db.commit() to db.session.commit() → no error, but it also doesn't actually save the change!
  • added SQLALCHEMY_COMMIT_ON_TEARDOWN = True to config.py → edits seem to save, but
    • when I re-edit the changed page, the body field has <p> tags?!?
    • and saving a new page doesn't work - db.add(node)'SQLAlchemy' object has no attribute 'add'
  • make it db.session.add(node) → saves

Nov06: what's with the <p> tags?

  • seeing it for a bunch of pages
  • hrm it's actually in the db!
  • but only in the top few - weird have I had that bug for awhile?
  • no actually only a couple pages - viewing the page is enough to make it happen!
  • I think it's the "magic" library logic, combined with me over-writing the object value (but not explicitly saving that back!)
  • made that a new variable, called it in template → now all good
  • but now realize that all my edited pages have modified=null
  • it's failing my 2 action value cases, falling into a last case I just use for remote-POST, and therefore not setting modified.
  • printing full response.form shows I have 2 action entries, one of which is empty?!?
  • can't find where this is coming from
  • maybe having a form param named action is a bad idea? (It hasn't been an issue in the past. But maybe doesn't play well somewhere in magic now....)
  • ah, it's not just that field! It's happening with another field which is defined as HiddenField in my forms.py class?!? (Since my template is old, all the fields in it are explicit


  • just make those field StringField instead of HiddenField - now setting Modified


  • back to Flask-User (diverted as of Nov01)
  • null username issue again/still
  • pondering: why do I even need a username if I'm going to have people user their email for authentication? Maybe I should just make the field optional in the db, and move on?
  • part of my confusion is USER_ENABLE_EMAIL and USER_ENABLE_USERNAME config settings... do you have to set just 1 or the other? Does me having u_e_u set to False cause the code to ignore the form field?
  • set them both to True → it worked! Record created with value filled in!
  • but what will happen if I try to log in?
  • try, but was using random password and didn't save it, so rejected! Or is it something else? Actually page crash with UnknownHashError: hash could not be identified
  • try Forgot-Password feature → SMTP Connection error: Check your MAIL_SERVER and MAIL_PORT settings
  • paste/fork Flask-Mail SMTP server settings → email sends, I receive it
  • click link in password-reset email → 'SQLAlchemy' object has no attribute 'commit'


  • change flask-user sql_db_adaptor.py to use db.session.commit() - had changed that when I was first trying to get flask-user working weeks ago.
  • password-reset → works, receive the email. Click, get form. Enter new password (and save it this time). Saves successfully! (Though it also says Your account has not been enabled.)
  • submit email/password - says Your account has not been enabled. I'm guessing that's because db users.active is null
  • log in to the other account, which I had manually set to active=t, and login is fine.
  • noting that the tutorial series I've been following users Flask-Login instead of Flask-User grrr
  • will play with mapping of user to space before getting around to Stripe integration
  • oh heck just realized my whole user-namespace thing is based on subdomains, so I need to map a domain to localhost, can't use Done.
  • found subdomain-route code
  • found filter AND syntax
  • realize that I never created any spaces upon creating the user.
  • Now wondering if I really need/want to have that.
  • if I force everyone to use /private and /wiki for their space names, it becomes unnecessary. (Or I could invent new defaults for everyone else, like /garden instead of /wiki, and handle mine as a special exception.)
  • hrm and spaces.path is redundant to users.subdomain
  • ah but spaces.id is foreign key in nodes.space_id, so I'd need to refactor that. And maybe I'll want to offer some people a 3rd space, etc.
  • so....
    • in registration form I should ask for path and title for each of 2 spaces, and create them when creating the user
    • and change my spaces.path to refer to the url-path rather than the subdomain
    • have to re-order my logic to look up path->space, and then get its spaces.privacy_type


  • had to put specific hostnames into my hosts file because it doesn't support wildcards for subdomains
  • had to put SERVER_NAME for domain into config (I think)
  • now I clearly have to go re-learn Flask-SqlAlchemy query/filter/join syntax
  • ah my tutorial series has a post that covers that
  • got the queries working (for view-1-page), now have to fix templates
  • have template content rendering, but wrapper bits are a mess - I think my static route handling fell apart in the refactor, though I thought I would have seen that affect sooner....
  • seems like static-path URLs aren't getting handled - connection refused - http://flux.garden:5000/static/bill2009_152.png - or, if I add a subdomain, then it gets picked up by one of my routes!
  • oh derp - when I added the subdomain cases to my hosts file, I left out the raw-domain case! So once I added that line to my hosts file, all good!


  • hit a private page → NameError: global name 'is_anonymous' is not defined. Derp, that's User.is_anonymous
  • now recognizing have to login, redirects to /user/sign-in but that is still within the subdomain, so it gets grabbed by the route and fails - need to redirect to the raw-domain. Ah, should use url_for.... ah, arg for that needs to be in quotes. Now good!
  • tk - redirect or at least link to source page after successful login
  • reload private page - get redirected to sign-in then instantly redirected to sign-in-successful page
  • ah, it's not User.is_anonymous, it's current_user.is_anonymous - I thought that smelled wrong. Now it's good.
  • now get edit link to render. Change to use is_authenticated, generates link but it's wrong. Running into issues with using subdomain model.
  • include subdomain in call to render_template() → link in template url_for('node_edit', page_url=page_url, path=path, subdomain=subdomain) rendered to http://flux.garden:5000/private/2019-09-05-Jangly/edit?subdomain=webseitz
  • posted to stackoverflow.


  • go no answer to url_for issue, so just making my own relative URL reference. Works.
  • hit edit link → getting matched to the page-view route instead of page-edit. Because haven't added the subdomain handler to the edit route+function. Also copy in other bits of refactoring from the page-view function.
  • hit edit → page container, but no form
  • rip out logged_in() conditional, fix functions to for new args, etc. → get form
  • submit edit → Saved! But redirect failed, added subdomain again
  • (and various other tweak) → edit saves, redirects to page-view all good
  • did the new-page variation → worked!
  • on to FrontPage: couple tweak → worked!
  • (actually, not true, it's not checking user yet, it's still the old logic there)


  • hmm, I'm going to manually create the 2 space records for user2, make sure the page view/edit bits are working, then hit the FrontPage. (see Nov16 notes)
  • created spaces records - going to have issues with setting ID-increment-start-point for this table, too


  • doing page-view - seems like favicon.ico is getting picked up by routes other than static, even though other static images aren't having that issue, and this file wasn't an issue with user_id=1.
  • Feels like I was failing-lucky before. Need to add url rule - but it's unclear where that goes (I had this in my old structure).
  • tried public/outer page-view (realized I was hitting a private, which adds complexity) - discovered Space lookup needed a lower() wrapper, fixed that.
  • now getting favicon error, but a different one, triggered by cascading redirects - - [23/Nov/2020 11:12:29] "GET /favicon.ico HTTP/1.1" 308 - - - [23/Nov/2020 11:12:29] "GET /favicon.ico/ HTTP/1.1" 308 -
[2020-11-23 11:12:29,086] ERROR in app: Exception on /favicon.ico/FrontPage [GET]
  • yikes cascading stupid, realize had space.privacy_type set to the path, not the privacy_type
  • then realized that the new user was id=16 not id=2 so the spaces weren't assigned right
  • now getting redirected to sign-in page
  • more derp I had set the wrong privacy_type for the space. Now rendering the page.
  • now realize I'm checking the user is logged in, but not that they own that space!


  • check that user owns page to view-private-page
  • have editing working in appropriate cases also
  • trying FrontPage variations
  • my outer FrontPage loads fine (after tweaks)
  • new-space outer FrontPage: nope it expects sidebar pages to exist already
  • created fallback empty-strings → works! (and sidebar-items link to page-view/edit url to create)
  • new-space inner FrontPage also works
  • hmm have some things hitting a domain-top-root - What will be there? A pitch? A list of (public) spaces? Both? OK, had commented out a route for that, put it back, tweaked that page to remind myself of what goes there, and how it fits into various flows....
  • next: RSS feed... working!
  • next: title-search
  • then tk: user-id=1 stuff has to be moved from global into something else....

Edited: |