WebSeitz/wiki

(2020-10-12) Building user management in WikiFlux

Building user management in WikiFlux so it will be multi-user. And putting in Stripe subscription setup.

What kinda flow do I want?

  • self-service to create
    • create "user"
    • then they log in
    • then pay, which then activates user which allows actual creation/editing (and links/redirects into the space)
  • login: hit link on any page, submit, get redirected back to self? Or root, I'm lazy.
  • view-vs-edit, privacy, etc.
    • can view public page anonymously, have login link in menu as today.
    • when viewing private page, or rendering Edit link for a view-public-page, have to check that the user is the owner of that space!

Oct12

  • pip install Flask-User → success
  • going to have to manually alter the users table schema.
    • am I going to use the role feature? Or do something nastier to limit admin rights to my 1 user?
    • probably keep it simple to start
    • for users table, make a SQL script I can run later in prod. Actually 2 bits, because have to modify the existing record before adding constraints. They're saved in file, but I ran them in command-line.
  • edit models.py
  • launch app →
class User(Base, UserMixin):
 NameError: name 'UserMixin' is not defined
  • made some tweaks, changed to
    'No Flask-SQLAlchemy, Flask-MongoEngine or Flask-Flywheel installed and no Pynamo Model in use.'\
flask_user.ConfigError: No Flask-SQLAlchemy, Flask-MongoEngine or Flask-Flywheel installed and no Pynamo Model in use. You must install one of these Flask extensions.
  • so it's not seeing the db connection. I wonder if I need to change my other code to use db = SQLAlchemy(app) but that probably requires a bunch of other changes. Maybe need to find some more example Flask apps, conform my code?
  • Oct18 hacked the flask-user db_manager code to assume it's sqlalchemy

Oct18

  • added email-config values - use bill@simplest-thing.com
  • it didn't like my secret_key, I had to muck around to satisfy it
  • now app is running!
  • paste in code for /members route
  • hit /members get sign-in page; enter info I had pushed into db before → get error, derp I added column email_confirmed should be email_confirmed_at
  • ALTER TABLE to fix column; hit again → UnknownHashError: hash could not be identified hmm not lots of help there
  • try registering new account bill@simplest-thing.com → AttributeError: 'scoped_session' object has no attribute 'session'
  • hack couple lines in flask-user to turn db.session. into db.
  • submit new-user again → fails because flask-user isn't creating id - null value in column "id" of relation "users" violates not-null constraint
  • while my code refers to field as auto-increment, nothing in the schema creates that.
    • hmm should I use field type serial? How convert to that? Nope, user more SQL-compliant identity/generated
    • once I settle on type, I can probably drop the existing id field then add the new type
    • decide on this - create sequence

Oct19

  • fix schema ^^
  • submit again - realize form is missing lots of fields I made required. Do I really want the user to have to enter them all to create an account? Probably....
  • derp they have a built-in template, I'll try the standard syntax to use that template!

Oct20

  • the built-in template only uses the couple fields, too. Gonna make local copy and add my fields. No still seems to be using their form.
  • change my route to /register, now get an error about form object for hidden_tag - research says that's for WTForms which is for CSRF protection, so I should be doing that anyway.
  • step back a sec - confirm app is working fine still with old hard-coded admin login, so I have a safe base to work form. So going to put in WTF/WTForms first. Get that working with old-model, then move forward.
  • pip install flask-wtf → I think it was automatically installed by flask-user
  • put in some code, but template is still unchanged.... it works
  • add in form.validate_on_submit() part - dies hard

Oct21

  • changed it to just use validate yeah it fails in there
  • ah probably forms class has to have fields matching what's in my manual form derp
  • still failing
  • derp had to put in {{ form.csrf_token }} in my form, boom!
  • (also confirmed making-new-page works)
  • next - back to Flask-User

Oct22

  • got to next error, I need to define the form class
    • I can extend the RegisterForm class (to add fields), or copy/paste/fork. I'm going to try extending.
  • tried forking, couldn't figure out how to import validators to call, etc. so went back to extending
  • finally got form to load
  • but have no method to submit yet

Oct23

  • before I add any user records, I need to get that serial/generated thing working.
  • ok it's done (did I just do it, or the other day?) - did pgdump of schema
  • add more fields to schema, model, form, field
  • next: POST logic
  • unclear from examples how much is manual to get the form data, and then save to db
  • try to see if validate_and_submit() does the job, appear not.
  • ah, key method is populate_obj()
  • but fail: was a class (models.User) supplied where an instance was required? - just needed parens
  • now running but not inserting

Oct25

  • ah print user says the populate_obj() call isn't actually doing anything....
  • oh derp actual problem was that I wasn't calling commit!
  • now have issue with username is empty, despite form... (which then triggers uniqueness issue)

Oct26

  • try ripping out bits of form logic, still no change. Other fields (both built-in and added-by-me) are handled fine.

Oct27

  • realize I wasn't following the right approach to customizing form templates. Move my custom form, change code to map to it. Form loads, but now submit passes all null data.
  • derp some of the doc pages I've been using are for 0.6 instead of 1.0!
    • different code change
    • nope all still null!
  • uncommented form.populate_obj(user) - now getting fields except for username again
  • trying adding username to MyRegisterForm() - didn't help, took it back out
  • asking for help in github

Nov01

  • wondering if I should step back and align my code's structure with a more "standard" Flask model. But what the heck is that?
  • Do I have to adopt Blueprints?

Plan: I'm going to suck it up and upgrade to current Flask, then restructure in line with this

Nov03

  • pip install --upgrade Flask no complaints. Runs fine!
  • tutorial puts a venv inside the app's directory, then an app in there. I'm not putting the venv down at that level. But I will make an app subdirectory of my app's directory
  • make wikiweb.py above app with just from app import app
  • make __init__.py inside app with the app = Flask(__name__) and similar bits
  • rename old core file wikiweb.py to routes.py. Rip out the main caller bits, swap in from app import app for past app-import bits.
  • set environment variable from command-line
  • flask run → flask_user.ConfigError: Config setting USER_EMAIL_SENDER_EMAIL is missing. even though it's there in the routes.py file. Should probably move some config stuff around.
  • so jump ahead to section 3 - make config.py above /app/
  • flask run → launches! Hit page - works!
  • go back to section1, do pip install python-dotenv and make .flaskenv file. Still runs
  • section2: I already have a templates subdirectory with templates, and they are working since my pages are loading.
  • section3: I have WTForms, but maybe now's a good time to play with that directly for a non-user/login form...
  • create new page, leave body empty, submit → "Internal Server Error", see my form.validate() was false, with form.errors = {'body': [u'This field is required.']}

Nov04

  • ah, I see, they use validate_on_submit() so that if validation fails they can treat it like a GET and deliver the form. Would need nice error handling, but that kinda makes sense....
  • tweak to use that → happy paths work; missing-field silent (to user) but doesn't explode
  • I already had flash-messages in my layout template, now call flash if validate fail

Nov05

  • section iv - databases
  • switch to db instead of db_session, stop importing the database.py file that calls scoped_session, move config bits into config.py
  • launch → get error on user_manager = CustomUserManager(app, db, User) because db is not defined - suspect this is part of the kludging I did to get flask-user working, so will go back to their raw instructions....
  • follow their starter-app organization - move UserManager bits into __init__.py
  • now launches. Hit page → db not defined - still need a from app import db in routes.py
  • then need to comment out the db shutdown/remove bits
  • now working - to view page
  • edit → 'SQLAlchemy' object has no attribute 'commit'
  • change all the db.commit() to db.session.commit() → no error, but it also doesn't actually save the change!
  • added SQLALCHEMY_COMMIT_ON_TEARDOWN = True to config.py → edits seem to save, but
    • when I re-edit the changed page, the body field has <p> tags?!?
    • and saving a new page doesn't work - db.add(node) → 'SQLAlchemy' object has no attribute 'add'
  • make it db.session.add(node) → saves

Nov06: what's with the <p> tags?

  • seeing it for a bunch of pages
  • hrm it's actually in the db!
  • but only in the top few - weird have I had that bug for awhile?
  • no actually only a couple pages - viewing the page is enough to make it happen!
  • I think it's the "magic" library logic, combined with me over-writing the object value (but not explicitly saving that back!)
  • made that a new variable, called it in template → now all good
  • but now realize that all my edited pages have modified=null
  • it's failing my 2 action value cases, falling into a last case I just use for remote-POST, and therefore not setting modified.
  • printing full response.form shows I have 2 action entries, one of which is empty?!?
  • can't find where this is coming from
  • maybe having a form param named action is a bad idea? (It hasn't been an issue in the past. But maybe doesn't play well somewhere in magic now....)
  • ah, it's not just that field! It's happening with another field which is defined as HiddenField in my forms.py class?!? (Since my template is old, all the fields in it are explicit

Nov11

  • just make those field StringField instead of HiddenField - now setting Modified

Nov12

  • back to Flask-User (diverted as of Nov01)
  • null username issue again/still
  • pondering: why do I even need a username if I'm going to have people user their email for authentication? Maybe I should just make the field optional in the db, and move on?
  • part of my confusion is USER_ENABLE_EMAIL and USER_ENABLE_USERNAME config settings... do you have to set just 1 or the other? Does me having u_e_u set to False cause the code to ignore the form field?
  • set them both to True → it worked! Record created with value filled in!
  • but what will happen if I try to log in?
  • try, but was using random password and didn't save it, so rejected! Or is it something else? Actually page crash with UnknownHashError: hash could not be identified
  • try Forgot-Password feature → SMTP Connection error: Check your MAIL_SERVER and MAIL_PORT settings
  • paste/fork Flask-Mail SMTP server settings → email sends, I receive it
  • click link in password-reset email → 'SQLAlchemy' object has no attribute 'commit'

Nov16:

  • change flask-user sql_db_adaptor.py to use db.session.commit() - had changed that when I was first trying to get flask-user working weeks ago.
  • password-reset → works, receive the email. Click, get form. Enter new password (and save it this time). Saves successfully! (Though it also says Your account has not been enabled.)
  • submit email/password - says Your account has not been enabled. I'm guessing that's because db users.active is null
  • log in to the other account, which I had manually set to active=t, and login is fine.
  • noting that the tutorial series I've been following users Flask-Login instead of Flask-User grrr
  • will play with mapping of user to space before getting around to Stripe integration
  • oh heck just realized my whole user-namespace thing is based on subdomains, so I need to map a domain to localhost, can't use 0.0.0.0. Done.
  • found subdomain-route code
  • found filter AND syntax
  • realize that I never created any spaces upon creating the user.
  • Now wondering if I really need/want to have that.
  • if I force everyone to use /private and /wiki for their space names, it becomes unnecessary. (Or I could invent new defaults for everyone else, like /garden instead of /wiki, and handle mine as a special exception.)
  • hrm and spaces.path is redundant to users.subdomain
  • ah but spaces.id is foreign key in nodes.space_id, so I'd need to refactor that. And maybe I'll want to offer some people a 3rd space, etc.
  • so....
    • in registration form I should ask for path and title for each of 2 spaces, and create them when creating the user
    • and change my spaces.path to refer to the url-path rather than the subdomain
    • have to re-order my logic to look up path->space, and then get its spaces.privacy_type

Nov17

  • had to put specific hostnames into my hosts file because it doesn't support DNS wildcards for subdomains
  • had to put SERVER_NAME for domain into config (I think)
  • now I clearly have to go re-learn Flask-SqlAlchemy query/filter/join syntax
  • ah my tutorial series has a post that covers that
  • got the queries working (for view-1-page), now have to fix templates
  • have template content rendering, but wrapper bits are a mess - I think my static route handling fell apart in the refactor, though I thought I would have seen that affect sooner....
  • seems like static-path URLs aren't getting handled - connection refused - http://flux.garden:5000/static/bill2009_152.png - or, if I add a subdomain, then it gets picked up by one of my routes!
  • oh derp - when I added the subdomain cases to my hosts file, I left out the raw-domain case! So once I added that line to my hosts file, all good!

Nov18

  • hit a private page → NameError: global name 'is_anonymous' is not defined. Derp, that's User.is_anonymous
  • now recognizing have to login, redirects to /user/sign-in but that is still within the subdomain, so it gets grabbed by the route and fails - need to redirect to the raw-domain. Ah, should use url_for.... ah, arg for that needs to be in quotes. Now good!
  • tk - redirect or at least link to source page after successful login
  • reload private page - get redirected to sign-in then instantly redirected to sign-in-successful page
  • ah, it's not User.is_anonymous, it's current_user.is_anonymous - I thought that smelled wrong. Now it's good.
  • now get edit link to render. Change to use is_authenticated, generates link but it's wrong. Running into issues with using subdomain model.
  • include subdomain in call to render_template() → link in template url_for('node_edit', page_url=page_url, path=path, subdomain=subdomain) rendered to http://flux.garden:5000/private/2019-09-05-Jangly/edit?subdomain=webseitz
  • posted to stackoverflow.

Nov20

  • go no answer to url_for issue, so just making my own relative URL reference. Works.
  • hit edit link → getting matched to the page-view route instead of page-edit. Because haven't added the subdomain handler to the edit route+function. Also copy in other bits of refactoring from the page-view function.
  • hit edit → page container, but no form
  • rip out logged_in() conditional, fix functions to for new args, etc. → get form
  • submit edit → Saved! But redirect failed, added subdomain again
  • (and various other tweak) → edit saves, redirects to page-view all good
  • did the new-page variation → worked!
  • on to FrontPage: couple tweak → worked!
  • (actually, not true, it's not checking user yet, it's still the old logic there)

Nov22

  • hmm, I'm going to manually create the 2 space records for user2, make sure the page view/edit bits are working, then hit the FrontPage. (see Nov16 notes)
  • created spaces records - going to have issues with setting ID-increment-start-point for this table, too

Nov23

  • doing page-view - seems like favicon.ico is getting picked up by routes other than static, even though other static images aren't having that issue, and this file wasn't an issue with user_id=1.
  • Feels like I was failing-lucky before. Need to add url rule - but it's unclear where that goes (I had this in my old structure).
  • tried public/outer page-view (realized I was hitting a private, which adds complexity) - discovered Space lookup needed a lower() wrapper, fixed that.
  • now getting favicon error, but a different one, triggered by cascading redirects
127.0.0.1 - - [23/Nov/2020 11:12:29] "GET /favicon.ico HTTP/1.1" 308 -
127.0.0.1 - - [23/Nov/2020 11:12:29] "GET /favicon.ico/ HTTP/1.1" 308 -
[2020-11-23 11:12:29,086] ERROR in app: Exception on /favicon.ico/FrontPage [GET]
  • yikes cascading stupid, realize had space.privacy_type set to the path, not the privacy_type
  • then realized that the new user was id=16 not id=2 so the spaces weren't assigned right
  • now getting redirected to sign-in page
  • more derp I had set the wrong privacy_type for the space. Now rendering the page.
  • now realize I'm checking the user is logged in, but not that they own that space!

Nov24

  • check that user owns page to view-private-page
  • have editing working in appropriate cases also
  • trying FrontPage variations
  • my outer FrontPage loads fine (after tweaks)
  • new-space outer FrontPage: nope it expects sidebar pages to exist already
  • created fallback empty-strings → works! (and sidebar-items link to page-view/edit url to create)
  • new-space inner FrontPage also works
  • hmm have some things hitting a domain-top-root - What will be there? A pitch (SplashPage)? A list of (public) spaces? Both? OK, had commented out a route for that, put it back, tweaked that page to remind myself of what goes there, and how it fits into various flows....
  • next: RSS feed... working!

Nov28

  • title-search tweak → working

Nov29

  • space-owner-id=1 stuff has to be moved from global into something else....
  • focus on view-node route first → working (though style doesn't look right for other users, have to dig in after doing other routes)
  • did other page → good

Dec01

  • next - styling that looks wrong - different font from the body downward, though the H1 looks the same.
  • did view-source of each, pasted into HTML files did diff on them, found only the expected differences.
  • checked Safari as well as Chrome and got same result. Grrr
  • Inspect element - the "wrong" style is getting e-content style from default.css, while the normal/expected is getting that overridden by cherwell.css
  • oh derp, missed that line was buried in a chunk of "expected differences", for some reason that line was adjacent to a chunk of IndieWeb stuff I'm commenting out for non-me users. Moved → now good!

Dec01

  • create stripe.com account
  • plan to use Checkout with subscriptions - use their hosted forms

What's left?

  • Stripe → activation flow
    • including creation of spaces - need to add fields to form and store them. Do that in the initial register form, or separate/later? Probably collect the data at register form, use them in creating the spaces, so don't need to store separately, and then just activation happens after Stripe.
    • hrm suspect I'm checking various things about space/user, but not is_active
  • auto-post to WikiGraph service/db for Visible Backlinks. (Which belongs on a page of its own.)

Dec11

  • realize that I had never un-borked the registration code. It still had the db/session mess. Now fixed.
  • add fields for 2 spaces to the registration form. Add add-space logic. Submit user → creates user, but no spaces.
  • derp need to add that to models import
  • did that → still not creating spaces
  • realized I should use form.inner_path instead of user.inner_path
  • noticing I'm doing things like db.session.add(outer) (and commit) instead of outer.save() - so try changing it. No difference.

Dec13

  • turn on debug mode, noticing that my print statements haven't been doing anything the last couple days.... Nope still not print ouput.
  • But I do notice a key field I'd left out in setting. Add that → nope still not enough.
  • I wonder whether my psql server has a nice log to view...
  • ask the db for log location. Find log, but no entries since Nov23?!? Probably not turned on since I moved to new machine. Turn logging on. Resort to log_statement. Stop/start service. Nope, no insert-space queries - so there's not an error there that's not getting passed back.
  • after a bit more flogging, post question

Dec14

  • got a suggestion in StackExchange, but it didn't help. Trying in discord

Dec15

  • tachyondecay at discord noted the key relationship line in my models.py was commented out. Probably during early-state problems. Uncommenting, plus tweaking: spaces = relationship('Space', backref='owner', lazy=True). Also adding SqlAlchemy_echo to config. Nope still only inserting user.
2020-12-15 19:49:54,563 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.username AS users_username, users.subdomain AS users_subdomain, users.email AS users_email, users.email_confirmed_at AS users_email_confirmed_at, users.password AS users_password, users.active AS users_active, users.first_name AS users_first_name, users.last_name AS users_last_name 
FROM users 
WHERE users.username ILIKE %(username_1)s 
 LIMIT %(param_1)s
2020-12-15 19:49:54,563 INFO sqlalchemy.engine.base.Engine {'param_1': 1, 'username_1': u'Test14'}
2020-12-15 19:49:54,588 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.username AS users_username, users.subdomain AS users_subdomain, users.email AS users_email, users.email_confirmed_at AS users_email_confirmed_at, users.password AS users_password, users.active AS users_active, users.first_name AS users_first_name, users.last_name AS users_last_name 
FROM users 
WHERE users.email ILIKE %(email_1)s 
 LIMIT %(param_1)s
2020-12-15 19:49:54,588 INFO sqlalchemy.engine.base.Engine {'email_1': u'fluxent+14@gmail.com', 'param_1': 1}
2020-12-15 19:49:54,855 INFO sqlalchemy.engine.base.Engine INSERT INTO users (username, subdomain, email, email_confirmed_at, password, active, first_name, last_name) VALUES (%(username)s, %(subdomain)s, %(email)s, %(email_confirmed_at)s, %(password)s, %(active)s, %(first_name)s, %(last_name)s) RETURNING users.id
2020-12-15 19:49:54,855 INFO sqlalchemy.engine.base.Engine {'username': u'Test14', 'first_name': u'Test14', 'last_name': u'Test14', 'email_confirmed_at': None, 'password': '$2b$12$o6A8jDSA6vwsBYGqrUMHQO7uFzqovXqRcP3Pfv5Ah7NawcYY86aRa', 'active': True, 'subdomain': u'Test14', 'email': u'fluxent+14@gmail.com'}
2020-12-15 19:49:54,857 INFO sqlalchemy.engine.base.Engine COMMIT
2020-12-15 19:49:54,858 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
2020-12-15 19:49:54,858 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.username AS users_username, users.subdomain AS users_subdomain, users.email AS users_email, users.email_confirmed_at AS users_email_confirmed_at, users.password AS users_password, users.active AS users_active, users.first_name AS users_first_name, users.last_name AS users_last_name 
FROM users 
WHERE users.id = %(param_1)s
  • FYI here's current libraries.
appdirs==1.4.4
configparser==4.0.2
contextlib2==0.6.0.post1
distlib==0.3.1
filelock==3.0.12
httplib2==0.18.1
importlib-metadata==2.0.0
importlib-resources==3.0.0
oauth2==1.9.0.post1
pathlib2==2.3.5
scandir==1.10.0
singledispatch==3.4.0.3
six==1.15.0
typing==3.7.4.3
virtualenv==20.0.32
zipp==1.2.0
  • after lots of thrashing, it seems most likely that my register method isn't even being called, but that the default Flask-User view is being used! Derp of the year. Really questioning whether I want to keep this magic, or switch to Flask-Login...

Dec18

  • so what are my options moving forward?
    1. rip out Flask-User and go with Flask-Login
    2. edit the Flask-User register view code in-place (which could be issue when I build something unrelated)
    3. copy/paste the root code in to my CustomUserManager and modify it there
  • I'm leaning toward opt3. Ok let's do this.
  • have to add some from flask import bits
  • ah, sequence issue on spaces.id - do alter sequence spaces_id_seq restart with 5 - sets last_value=5 - which will actually be the next value
  • submit → global name 'signals' is not defined - add signals to from flask import line
  • submit → AttributeError: 'module' object has no attribute 'user_registered' - also associated with signal. Decide to just delete that line, since I don't think I'll need signals.
  • yes, everything worked!
  • but want it to finish by redirecting to a different/special/login page
  • set USER_AFTER_REGISTER_ENDPOINT = 'user.login' in config
  • and make a home page post-login to check user status (paid, confirmed). so set USER_AFTER_LOGIN_ENDPOINT = 'home'

Dec19

  • make home page
  • notice all my users have active=t, why is that being set automatically?
    • I want to set user to active iff (a) email_confirmed and (b) paid
  • change a user to active=f, login → Your account has not been enabled.
  • this seems baked deep into Flask-Login, not just Flask-User. I could try to change that, but it smells like a bad idea.
  • So I'll just adjust my page-flow so that Login is the last step.
  • Actually, I don't think I'll have to change flow, I'll just add some explanation to the Login page.

Dec20

  • changed 'home' node to just redirect them to their Inner space FrontPage (if active, etc.)
  • added link from inner to outer space in FrontPage
  • next: for private pages that trigger login, pass that page along so that user gets redirected to original-request after login. Argh, next argument from Flask-Login doesn't nicely handle subdomains. Or maybe I'm doing it wrong. See discussion.
    • see also Nov20 notes above around url_for and subdomains
    • for that issue, see code around app.url_build_error_handlers.append(external_url_handler)
    • but going to table this for now, it's not crucial
  • next: trigger Stripe payment

Dec21

  • install CLI for webhook testing
  • up to Step 4 Create a Checkout Session
  • paste and tweak route for create_checkout_session, create routes for pay_success and pay_cancel
    • going to put those templates into /flask_user/
  • copy entire script.js into static - but what calls it? Does Stripe just assume it's at root? Might have to move it.
  • copy the rest of the bits in that page, but don't understand any of it, and suspect it doesn't fit together.
    • though I do see the bits in the webhook for even when invoice_paid etc
      • should probably be storing some payment info in the user record (or in a payments table), and have some sort of method for updating user.active
  • looking for some Flask-specific docs/examples
    • Stripe has a legacy page made obsolete with Checkout in Apr'2019, so probably not that
    • there's a video
    • but let's try this page, following "fixed price" model.
  • starting this off by moving to .env file for environment variables, and use python-dotenv to load them
    • more precisely, use .flaskenv for Flask CLI configuration, like FLASK_APP etc (find put those in our .flaskenv file in article), and use .env file for passwords and such
    • but I don't think I need to put my price_id in the env file because of I have 2 prices, and I think that all happens automatically
    • stripe doc refers to script.js, this article has main.js which seems to be the same (and is in static), so I'm renaming mine to stripe.js
    • stopping to read ahead at the Authentication section - decide not going to add any db stuff until I get a skeleton working.
  • back to core Stripe documents. Also going looking for flow-diagrams
  • duh realize that Stripe doesn't offer a product page or shopping cart, just the payment. So I need a page listing the prices, but a button for each.

Dec22

  • smells like if user is not active then he doesn't get logged in, so no current_user defined, etc. Which doesn't work for passing along email to Stripe, etc. Which makes it smell like
    • I should just let new accounts be set active=True
    • I should set USER_ALLOW_LOGIN_WITHOUT_CONFIRMED_EMAIL=False to put that block in place
    • then I need a post-Login page which checks payment status, gives them subscription buttons if not paid, etc. (if permitted, then redirect them to private space) - I'll call it home
    • and I'll have a users.permitted field to track whether confirmed and current-paid
      • which circles back to how I track payment status with Stripe. They send invoice.paid or invoice.payment_failed events to my webhook
      • I should probably store every payment in a table. Or, really, every payment-event.
      • when I receive a checkout.session.completed event I set users.permitted=True
      • when I receive invoice.paid I set users.permitted=True
      • when I receive invoice.payment_failed I set users.permitted=False - I don't have to track when last-payment made, when due for next payment, etc., Stripe does all that for me.
      • when receive customer.subscription.deleted (means they canceled), set users.permitted=False
      • if permitted=False and confirmed=True then I check my payment_events table to give user feedback
      • do I store the Stripe customerId on the users table or the payment_events table? This tutorial makes a stripe_customers table - but won't I have 1:1 from User to StripeCustomer? And both 1:1 to Subscription? Or maybe if someone changes payment-model they get a different SubscriptionId? I guess I'll follow that model: table payment_customers
      • but I still need a table to store events, I think.
        • my own id, plus a source, plus a source_event_id_id for Stripe's id (planning ahead); plus a SubscriptionId or CustomerId?
        • maybe a status or is_handled? Nah, probably won't have a separate/async process, so everything will be handled as it's received
        • a datestamp
        • an event_type field
        • a blob field to store whatever else comes, as JSON
        • duh realize I want this to be a general-purpose events table (so went back and added field source above

Dec26

  • update models.py, sql schema per above

Jan03

  • mind-reset: write Using Stripe Checkout For Your SaaS Subscription
  • try a user login (test-06): great, it rejects login because email not confirmed.
  • re-send confirmation email; it redirects me to root page rather than login, but I guess that's ok since next action is really to go find that email anyway.
  • click confirm link - it updates db, but redirects me to root again. Want to see if that's changeable... yep change
USER_AFTER_CONFIRM_ENDPOINT = 'user.login'
USER_AFTER_RESEND_EMAIL_CONFIRMATION_ENDPOINT = 'user.login'
  • yes, now after confirmation get redirected to /home (because have USER_AUTO_LOGIN = True)
  • right now my Home page is showing 1 Subscribe/Pay button with <button id="checkout"> and clicking it does nothing.
  • turn that into 2 buttons, with id monthly-btn and yearly-btn
  • in my .env file put MONTHLY_PRICE_ID and YEARLY_PRICE_ID API Ids that Stripe assigns
  • then in my routes.py I define @app.route("/setup")... def get_publishable_key(): to return those Price IDs
  • then modify stripe.js (which I renamed from their script.js) to fetch the appropriate price-id and redirect to Stripe.
  • and... I see 2 buttons... but clicking does nothing. Derp my home.html template needs to reference the 2 JavaScript files - I had only done this for login.html
  • and... still nothing. View-source shows the JavaScript lines, clicking on my local stripe.js loads that file fine....
  • I see /setup in the console getting called and returning the price-ids
  • look in JavaScript console: Uncaught (in promise) IntegrationError: Invalid value for Stripe(): apiKey should be a string. You specified: undefined. Ah, I was returning publicKey instead of publishableKey.
    • sidebar issue: console now notes that live integrations must use HTTPS/SSL which I don't have running on my server. tk
  • now not working because /create_checkout_session route is giving a 400
    • I think it's saying "global name 'CHECKOUT_SESSION_ID' is not defined" (I went into chrome dev/devtools, picked "network", then clicked on the red create-checkout-session)
    • realize the error is being hidden because all this is inside a try - move variable-setting bits outside, confirm issue is global name 'CHECKOUT_SESSION_ID' is not defined when trying to attach it to success_url
    • stop using url_for and use
success_url = app.config["SERVER_PROTOCOL"] + app.config["SERVER_NAME"] +'/pay_success?{CHECKOUT_SESSION_ID}'

→ works!

Feb06 getting my Linode server caught up

  • pip install --upgrade Flask → lots of warning about InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  • python is 2.7.6
  • doing nothing about it quite yet
  • pip install --upgrade pip → Successfully installed pip-21.0.1 (with same warnings)
  • do pip install pyOpenSSL ndg-httpsclient pyasn1, → sys.stderr.write(f"ERROR: {exc}").... SyntaxError: invalid syntax which is not surprising: pip 21.0 dropped support for Python 2.7 and Python 3.5. Hmm maybe I'll downgrade to pip 20.3.4... → invalid syntax
  • tried using python get-pip.py → invalid syntax?!?! Ah tried again with 2.7-specific flag, now seems ok (though still giving the original urllib3/ssl InsecurePlatformWarning)
  • so tried again pip install pyOpenSSL ndg-httpsclient pyasn1 - get warnings during the install, unclear whether it matters. Going to move forward....

Feb07

  • pip install python-dotenv → successful, but with InsecurePlatformWarning
  • pip install Flask-User → Couldn't find index page for 'Flask-Login' (maybe misspelled?)
  • pip install Flask-Login → Successfully installed Flask-Login-0.5.0
  • pip install Flask-User again → same error (2nd line is No local packages or download links found for Flask-Login - which is why I thought installing Flask-Login directly itself would solve the problem)
  • ls of dist-packages shows flask_login and Flask_Login-0.5.0.dist-info
  • try pip install Flask-User --no-dependencies → same error

Fork

Now what?

May14

  • start on webhook code
  • realize I don't want to call the endpoint /webhook, so I rename it.
  • otherwise, my code is a copy of this
  • let's launch the CLI and see if it works at logging events, before turning that into working stuff. See this page.
  • Do the stripe login pairing
  • Do stripe listen --forward-to flux2.garden:5000/stripewebhook → get secret, add to .env, add equiv to stripe_keys. Do flask run.
  • Pick a user with active and email_confirmed_at but not permitted, log in as that user. Get redirected to page saying "Note that your account isn't active until you have paid". Press a button... get Stripe payment page. Fill in fake number, submit... → see lots of events in CLI window. But they all have 404. Likewise the flask window shows 4 POST with 404 responses. Derp realized that stripe listen command was missing underscore in stripe_webhook. Quit, re-do forward-to, have same signing hook.
  • Do payment again. See 404 responses again (though at least the route is accurate).
  • Realize different parts of my code use variables webhook_secret and endpoint_secret - decide to stick with the latter.
  • Pay again → 6 successful POSTs almost immediately in the CLI window
2021-05-14 22:13:40   --> checkout.session.completed [evt_1IrDwaEl5VTchHIeCX0IZlgN]
2021-05-14 22:13:40  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrDwaEl5VTchHIeCX0IZlgN]
2021-05-14 22:13:41   --> customer.subscription.updated [evt_1IrDwaEl5VTchHIe0jI46uQ1]
2021-05-14 22:13:41  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrDwaEl5VTchHIe0jI46uQ1]
2021-05-14 22:13:41   --> invoice.payment_succeeded [evt_1IrDwaEl5VTchHIe2f0f2pKr]
2021-05-14 22:13:41  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrDwaEl5VTchHIe2f0f2pKr]
2021-05-14 22:13:41   --> payment_intent.succeeded [evt_1IrDwbEl5VTchHIekBQdxaz7]
2021-05-14 22:13:41  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrDwbEl5VTchHIekBQdxaz7]
2021-05-14 22:13:44   --> invoice.created [evt_1IrDwaEl5VTchHIewc6lt6F0]
2021-05-14 22:13:44  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrDwaEl5VTchHIewc6lt6F0]
2021-05-14 22:13:46   --> invoice.paid [evt_1IrDwaEl5VTchHIeqXyyT2vt]
2021-05-14 22:13:46  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrDwaEl5VTchHIeqXyyT2vt]
  • then 5 more a minute later
2021-05-14 22:22:30   --> checkout.session.completed [evt_1IrE59El5VTchHIenK6L6qR4]
2021-05-14 22:22:30  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrE59El5VTchHIenK6L6qR4]
2021-05-14 22:22:31   --> invoice.updated [evt_1IrE5AEl5VTchHIeOWNm1zC5]
2021-05-14 22:22:31  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrE5AEl5VTchHIeOWNm1zC5]
2021-05-14 22:22:31   --> invoice.updated [evt_1IrE5AEl5VTchHIeDEK3xmdR]
2021-05-14 22:22:31  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrE5AEl5VTchHIeDEK3xmdR]
2021-05-14 22:22:32   --> invoice.paid [evt_1IrE5AEl5VTchHIenIHYb4Jk]
2021-05-14 22:22:32  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrE5AEl5VTchHIenIHYb4Jk]
2021-05-14 22:22:34   --> customer.subscription.created [evt_1IrE5AEl5VTchHIe1PKCysGc]
2021-05-14 22:22:34  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IrE5AEl5VTchHIe1PKCysGc]
  • Flask window shows mix of "Unhandled event type" lines and a more-detailed data-dump for defined event types.
  • so ready to store these and update user-state

May15

  • reviewing that I have a PaymentCustomer table (1 row per customer-purchase, so at least for now just 1 row per paid customer) and Event table (for all these payment events).
  • see Dec22 notes above for which events to take action on (but I'm going to log them all)
  • how is payer identified in Stripe events?
    • related: I see http://flux2.garden:5000/pay_success?{CHECKOUT_SESSION_ID} in my flask console, which smells like I have an un-rendered variable somewhere... but looking at my test user I do see a real value in the URL, so maybe that logging is just early/wrong.
  • in checkout.session.completed
    • "customer": "cus_JUCUsNGO3AJZXx"
    • customer_email ah right you can't change that value over at Stripe, so I don't have to worry about it not matching.
    • "id": "cs_test_a1ANzAPUI9ELQVbB42i44pM4So7KkejRMyh1BQXW2SC6IcQm6dU6mdJvB8"
    • "subscription": "sub_JUCUJW8z7EwNdV"
  • in invoice.paid
    • "customer": "cus_JUCUsNGO3AJZXx" and customer_email
  • for now I'm just going to use the customer_email to look up my user, and update user.permitted=True, and save all the events to that table
  • pay → all 500s. Derp didn't import the models. Fix
  • pay → all 500s, different reason
  • at least 1 issue is that I'm taking the created timestamp from Stripe and trying to create Event object, but that wants DateTime passed, so have to convert. Actually, going to ignore what Stripe passes and just take the current-time. Also restructure some logging
  • pay → 500s. Notice first call was charge.succeeded, and customer_email was buried in a different level. Hmm could the issue be that the customer already exists from prior attempts? Yep they're there. Can delete customers from my dashboard, but not invoices. Delete all → see more webhooks come through, but they also got 500. Partially because the customer_email is just named email in some of these events, jfc. Even the customer ID is sometimes id and sometimes customer, are they really kidding me.
  • for a given object being returned, it can have sub-objects; at any level, the object's object attribute tells you the object type, and the id will be that object-type-id. If the object isn't the customer, then the customer attribute is the customer_id.
  • So it feels like I need to make sure to create that PaymentCustomer record to associate their customer_id with my user_id, then count on the customer_id being available, whether as customer or id.... It sounds like the first event from a new customer/purchase will be checkout.session.completed so that's where I should assume I'm creating a customer, and in the other events assume that I'm looking one up. (At least as a starting point - but I probably have to play it safe and always do a lookup first, then create if not found....

May16

  • so for event checkout.session.completed
"object": "checkout.session",
"customer": "cus_JUrckzVGNfP1UT",
"customer_email": "fluxent+hunter@gmail.com",
"subscription": "sub_JUrcPaHNfV8H92",
  • so that's where I'll create PaymentCustomer (if subscription doesn't already exist), and for all other cases I'll assume customer is customer_id, and look up the PaymentCustomer to get the user_id, and use that for everything.
  • pay → nope
2021-05-16 17:28:20   --> charge.succeeded [evt_1IrsRbEl5VTchHIe1mNO67I3]
2021-05-16 17:28:20  <--  [500] POST http://flux2.garden:5000/stripe_webhook [evt_1IrsRbEl5VTchHIe1mNO67I3]
2021-05-16 17:28:21   --> customer.created [evt_1IrsRbEl5VTchHIeWw8ZY97W]
2021-05-16 17:28:21  <--  [500] POST http://flux2.garden:5000/stripe_webhook [evt_1IrsRbEl5VTchHIeWw8ZY97W]
2021-05-16 17:28:22   --> customer.subscription.created [evt_1IrsRbEl5VTchHIe1daEpL8s]
2021-05-16 17:28:22  <--  [500] POST http://flux2.garden:5000/stripe_webhook [evt_1IrsRbEl5VTchHIe1daEpL8s]
(and many more) - all 500s
  • in charge.succeeded - pmt_customer_id = object['customer'].... TypeError: 'type' object is not subscriptable.... same in others
  • derp, had created data_object=data['object'], so changed that name to object
  • pay → still fail, ah I think that because earlier attempts created customer(s) on Stripe side, but PaymentCustomer failed to create on my db side, the order of events changes, and I don't have the record to look up.
  • I think I'm going to re-start by ignoring (not even logging) most events, and just handle the few called out by others.

May17

  • pay → lots of 200s but not all
2021-05-17 20:14:42   --> payment_method.attached [evt_1IsHWAEl5VTchHIegtef6HZL]
2021-05-17 20:14:42  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIegtef6HZL]
2021-05-17 20:14:42   --> customer.created [evt_1IsHWAEl5VTchHIeMKfCdiWZ]
2021-05-17 20:14:42  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIeMKfCdiWZ]
2021-05-17 20:14:42   --> invoice.updated [evt_1IsHWAEl5VTchHIePFPjhYMc]
2021-05-17 20:14:42  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIePFPjhYMc]
2021-05-17 20:14:42   --> invoice.finalized [evt_1IsHWAEl5VTchHIepEyR6y9j]
2021-05-17 20:14:42  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIepEyR6y9j]
2021-05-17 20:14:43   --> checkout.session.completed [evt_1IsHW9El5VTchHIe5Yn6H6cc]
2021-05-17 20:14:43  <--  [500] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHW9El5VTchHIe5Yn6H6cc]
2021-05-17 20:14:43   --> customer.subscription.updated [evt_1IsHWAEl5VTchHIef8vM3sXn]
2021-05-17 20:14:43  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIef8vM3sXn]
2021-05-17 20:14:43   --> invoice.paid [evt_1IsHWAEl5VTchHIem2zSNx4c]
2021-05-17 20:14:43   --> customer.subscription.created [evt_1IsHWAEl5VTchHIe6RoKt2cG]
2021-05-17 20:14:43  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIe6RoKt2cG]
2021-05-17 20:14:43  <--  [500] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIem2zSNx4c]
2021-05-17 20:14:43   --> invoice.payment_succeeded [evt_1IsHWAEl5VTchHIeFiELy5no]
2021-05-17 20:14:43  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IsHWAEl5VTchHIeFiELy5no]
(more 200s after)
  • (and there's no PaymentCustomer record, either)
  • derp I used == instead of = in one of the query.filter_by() clauses
  • also see an issue with inserting Event because of null id, which should be auto-incrementing.
  • try manual sql insert of Event - null id fail, so I must have the table set up wrong. Do alter column id ADD GENERATED for each.
  • pay → I see all 200s! But... I see no PaymentCustomer and no Events. Hmm, CLI smells funky (list of events seems too short), stop/restart it.
  • pay → I see all 200s, starting with a checkout.session.completed - and I see Events and PaymentCustomer! (though latter missing created datetime), and user.permitted=True! Success!
  • user who just paid is at http://flux2.garden:5000/pay_success?cs_test_a1s4YWKJvNAxyqRWjbVxp57ZweRu6B0OPC5PZSGl8O04KjPlCtEPr2jsJT so I'll want to alter that to add explicit link to Home (the link is there, but it's not obvious that's what you should do).
  • also the link to Home just goes to root, not to /home
  • click "Home" for root → that's the static true-root page
  • manually going to /home redirects to inner space nicely... hmm should I re-consider that, have page with links to inner and outer?

May18

  • add link to SignIn on SplashPage
  • realize that "inner" FrontPage already has a link to the outer site in upper right.
  • do I want to add the opposite to outer FrontPage if user is logged in? Done.
  • trying another unpaid user to confirm first-time experience
  • first get confirmation email blocked by gmail SMTP until I fix that
  • then new-user account gets "Your account has not been enabled." because user.active=f - need to fix that default. Hmm see Dec19 notes, something is wrong here...

May19

  • let's document new-user flow more carefully...
  • start at SplashPage, hit Register button, fill in form, hit Submit
  • get /user/sign-in page with msg "A confirmation email has been sent to fluxent+30@gmail.com with instructions to complete your registration."
  • record created with active=f, permitted=null, email_confirmed_at=null
  • email's link is to /user/confirm-email/<token>. Click it, get /user/sign-in with 2 messages
    • "Your email has been confirmed."
    • "Your account has not been enabled."
  • try to sign in anyway → get kicked back with "Your account has not been enabled."
  • options
    • set user.active=t upon creation
    • have email-confirmation lead to payment page... except I have that working through /home which wants them to be identified. Or does email-confirm identify them?
    • my config says USER_AFTER_CONFIRM_ENDPOINT = 'user.login' can I change that to home?
      • grr note current behavior not consistent with my Jan03 notes! And I still have USER_AUTO_LOGIN=True set!
  • just try changing that endpoint to home - I think it hit home, but that route checks current_user.is_anonymous and bumps user to sign-in
  • add USER_AUTO_LOGIN_AFTER_CONFIRM = True

May20

  • nope that didn't work either. But need to read more carefully about how it behaves.... nope not finding any info
  • just going to change things to set active=t upon creation
  • yes value is set. Hit confirm link in email → yes get home page with payment buttons. Pay → /pay_success page. That page has links
    • FluxGarden → root
    • "Home" → root
    • username → edit-profile
    • sign-out
  • want to change that Home link to /home - edit flask_user_layout.html → works
    • also want to change Company/year footer. Where is USER_CORPORATION_NAME set? I think there's a default buried in the code, just override in my config.py → all good
  • now look at /home
    • show the "Note that your account isn't active until you have paid" even though I have paid. I suspect my route checks some logic, and actually hitting this page requires meeting a couple requirements, 1 of which I lack - like is_authenticated
      • yes, first not-is-authentication → sign-in, elseif permitted → inner-frontpage, else home/payment
    • hrm a bit later it shows the sign-in page, which is more expected. But need to repeat that experiment....
  • also fix the home/payment page so it uses the user-pages template-container like it should.
  • login → get home/payment?!? Ah, I see, the Stripe CLI isn't actually working, so the payment didn't reach the webhook, so user.permitted is still null!

May24

  • draft SplashPage
  • re-try registration with CLI running → hrm looks like auto-login is working? But if I click Home, it says I haven't paid. Hmm all the webhook calls got 404. Derp copied wrong try at CLI call, which had wrong webhook route/url.
  • try again, still no good! Got 200s this time, but not the full set of transactions, so never set permitted
2021-05-24 21:45:20   --> invoice.created [evt_1IuqGhEl5VTchHIeJUUwNBCY]
2021-05-24 21:45:20  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IuqGhEl5VTchHIeJUUwNBCY]
2021-05-24 21:45:20   --> customer.subscription.created [evt_1IuqGhEl5VTchHIevKDew47G]
2021-05-24 21:45:20  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IuqGhEl5VTchHIevKDew47G]
2021-05-24 21:45:20   --> invoice.updated [evt_1IuqGhEl5VTchHIeyff2jfgI]
2021-05-24 21:45:20  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IuqGhEl5VTchHIeyff2jfgI]
2021-05-24 21:45:20   --> customer.subscription.updated [evt_1IuqGhEl5VTchHIeK8L49IIK]
2021-05-24 21:45:20  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IuqGhEl5VTchHIeK8L49IIK]
2021-05-24 21:45:20   --> invoice.updated [evt_1IuqGiEl5VTchHIeNcLR7ZHW]
2021-05-24 21:45:20  <--  [200] POST http://flux2.garden:5000/stripe_webhook [evt_1IuqGiEl5VTchHIeNcLR7ZHW]
  • so no checkout.session.completed - maybe need to delete old users?

May25

  • sounds like I need to update my Stripe CLI. Now how the heck do I do that?
  • after sniffing around it looks like I must have installed it using homebrew, since the CLI is inside Cellar and there's a INSTALL_RECEIPT.json file pointing to homebrew.
  • there doesn't seem to be a separate update command, so just install as if I didn't have it. Seems to do what I expected. But launching CLI again (after login/pairing) still gives me "A new version of the Stripe CLI is available..."
  • looks like the shortcut at /usr/local/bin/stripe still points to ../Cellar/stripe/1.5.5/bin/stripe - and there's no 1.6.0 inside Cellar/stripe
  • so just download the CLI source itself and untar, then manually copy into/over Cellar/stripe/1.5.5... (have to do the security-allow-unsigned Mac thing, but after that it launches fine, and doesn't alert me to update)

May26

  • run-through again
  • register; get "confirmation email was sent"; click link, get /home saying email confirmed, signed in successfully, note not active until you've paid
  • pay → /pay_success plus lots of 200 events
  • click Home → http://flux36.flux2.garden:5000/inner/FrontPage but We can’t connect to the server at flux36.flux2.garden. and console shows a 302
  • oh right, each hostname has to be entered individually to my hosts file (see Nov17 notes above). Reload → success! Plus db shows correct user record/fields plus 2 events added.
  • hrm FrontPage looks odd - it shows 1 item, MyPage, with content, even though obviously that can't exist. And if I click on it I get a weird header/title. So I must not be handling the empty-list well...
  • ah bad query, fix → now good
  • (also fix space-query for title search)
  • click on a sidebar page (which doesn't actually exist) → yep. Click Edit to edit/create, title is prefilled with current date instead of correct pagename. That's an old bug I'm finally going to fix now (from when I tried to create a "quick" way to start a new blogbit page)... fixed.
  • (finally do a git commit way overdue)
  • save that page edit/create - get weird response/redirect; go back to FrontPage, see it listed, but if I click on it get strange redirect-result. Grr looks like page was actually created in wrong space so why is it showing - I guess still more places where my ORM query isn't quite right.
  • Now seems fixed. git commit

May27

  • playing with WikiGraph for TwinPages and Backlinks.
  • add a space to db
  • create a page in that space matching one of my pagenames - yes TwinPages appears with link
  • insert a mentions into db - hmm doesn't seem to work
  • ah, in this case I have code checking just for my referer, so need to make that more flexible, will use the domain

May29

Jun02

  • now ready to try and get all this updated to production
  • plan
    • pull git
    • update .env for a change I had
    • run, see what breaks, give one piece at a time (I know there will be db structure changes to make, but I'll do them as-demanded)
    • that will still be using "test" credentials for Stripe, which is good for testing
    • then get production credentials in place
  • update: all I had to do was add the mentions table, and main stuff is working fine.
  • new user register → 502 bad gateway. Log says SMTP Connection error: Check your MAIL_SERVER and MAIL_PORT settings. Maybe I have that port blocked? No, it seems like Google is blocking it. Even though Google accepts it fine from my laptop.
    • actually main/initial error is Username and Password not accepted.
  • try using my Tuffmail instead. Get similar/different error. Try turning on TLS, no explicitly says that's not supported.
send: 'ehlo [127.0.1.1]\r\n'
reply: b'250-smtp.mxes.net\r\n'
reply: b'250-PIPELINING\r\n'
reply: b'250-SIZE 104857600\r\n'
reply: b'250-AUTH PLAIN LOGIN\r\n'
reply: b'250-AUTH=PLAIN LOGIN\r\n'
reply: b'250-ENHANCEDSTATUSCODES\r\n'
reply: b'250-8BITMIME\r\n'
reply: b'250-DSN\r\n'
reply: b'250 CHUNKING\r\n'
reply: retcode (250); Msg: b'smtp.mxes.net\nPIPELINING\nSIZE 104857600\nAUTH PLAIN LOGIN\nAUTH=PLAIN LOGIN\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nCHUNKING'
send: 'AUTH PLAIN AGJpbGxfZmx1eGVudC5jb20AeHh4\r\n'
reply: b'535 5.7.8 Error: authentication failed: authentication failure\r\n'
reply: retcode (535); Msg: b'5.7.8 Error: authentication failed: authentication failure'
send: 'AUTH LOGIN YmlsbF9mbHV4ZW50LmNvbQ==\r\n'
reply: b'334 UGFzc3dvcmQ6\r\n'
  • try Google again from droplet but with TLS turned on → smtplib.SMTPNotSupportedError: STARTTLS extension not supported by server
    • ah here's a pretty good spec, so revert to SSL/465, no TLS
send: 'ehlo [127.0.1.1]\r\n'
reply: b'250-smtp.gmail.com at your service, [159.65.175.127]\r\n'
reply: b'250-SIZE 35882577\r\n'
reply: b'250-8BITMIME\r\n'
reply: b'250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH\r\n'
reply: b'250-ENHANCEDSTATUSCODES\r\n'
reply: b'250-PIPELINING\r\n'
reply: b'250-CHUNKING\r\n'
reply: b'250 SMTPUTF8\r\n'
reply: retcode (250); Msg: b'smtp.gmail.com at your service, [159.65.175.127]\nSIZE 35882577\n8BITMIME\nAUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH\nENHANCEDSTATUSCODES\nPIPELINING\nCHUNKING\nSMTPUTF8'
send: 'AUTH PLAIN AGJpbGxAc2ltcGxlc3QtdGhpbmcuY29tAHh4eA==\r\n'
reply: b'535-5.7.8 Username and Password not accepted. Learn more at\r\n'
reply: b'535 5.7.8  https://support.google.com/mail/?p=BadCredentials 42sm2866032qtf.37 - gsmtp\r\n'
reply: retcode (535); Msg: b'5.7.8 Username and Password not accepted. Learn more at\n5.7.8  https://support.google.com/mail/?p=BadCredentials 42sm2866032qtf.37 - gsmtp'
  • I've even copied files back and forth again and done diffs to make sure they're right. Still no change.
  • also tried hitting https://accounts.google.com/DisplayUnlockCaptcha but that didn't help
  • filed support ticket and tweet.
  • hrm nmap shows those ports as closed, and iptables failed to change that. But I tried doing a telnet on 45 to smtp.gmail.com and it worked fine.

Jun04

  • hmm I see some people referring to smtp.googlemail.com so I'll try that → no difference
  • try TLS/587 instead of SSL → same rejection
  • add some logging in flask_mail.py → password ='xxx'???
  • mystery solved! I missed that my friend, in setting up my system, is using envfile_service instead of .env - and she had written some junk in there for passwords. So when I copied .env over it, boom it worked!
  • continue on to Stripe part, still in "test mode"
  • confirm email, go through Stripe, come back to home → still not active (which actually means users.permitted is null
  • yep events table is empty - I think I had to do a schema change there which I never did in production.... yep 2 cases of making id auto-increment, see May17 notes above
  • new user → same outcome. Derp I must need to install the Stripe CLI on the server, too (or check if there's some other way to run testing once you're out on the net...)

Jun05

  • can I test on server without CLI? Boy these Stripe docs really suck.
  • ah had to register my webook. Then, in that webhook, I can "send test webhook" and pick an event-type to send → 502 error. Doesn't even touch any of my code, and log error is TypeError: The view function did not return a valid response. The return type must be a string, dict, tuple, Response instance, or WSGI callable, but it was a SignatureVerificationError.
  • this tutorial catches that exception, but doesn't explain what would be causing it
  • ah, pretty sure I need to get the new/different (from the CLI) signature for this webhook registration
  • sent another test → 502 → but see what was passed - it has mostly null values, so my webhook lookup finds no user and chokes. But at least I know the plumbing is working.
  • do fake client signup → success! And see expected records in events and payment_customers.
  • switch to live mode in Stripe, get my 2 keys and endpoint-secret, update my envfile, restart.
  • realize that even the Products you define in Stripe in testing are only for testing, and to manually recreate them in "live", then get those new keys to put in envfile.
  • how do you know when you're really live? When someone actually signs up and pays?
  • Phil Jones registered, and I activated him for free as alpha tester. Of course he's finding bugs.

Jun07

  • setting up free trial period
    • seems easy enough
    • yes, confirmed in UI in stripe payment page
    • but still got invoice.paid event right away?
  • announced to telegram list

Jun09

  • discover another bug around using the correct space

Jun10

  • get email about webhook failing, but it turns out that was from the test-mode - seems like that should be more clear in email....
  • decide to pay myself to walk through final bits live
    • payment successful
    • when done, URL is /pay_success? with no urg-arg, and it turns out the Manage Billing button doesn't do anything - is it because I don't have that url-arg?

Jun11

  • setting up Stripe customer portal...
  • add privacy and terms pages
  • configure portal
  • integrate? (need to check my old miro sitemap/flow)
  • added editing/saving screenshots to SplashPage
  • created short getting-started video, added it to Getting Started With FluxGarden
  • also add some Bootstrap container tags for blocks of text for some padding.
  • back to portal...
  • aim to use /pay_success as the home for this?
  • ah, had def customer_portal() but no route, and had to add that route to the "Manage Billing" button
  • but getting a JS error...

Jun12

  • realize should use the current_user to drive the portal link, so change code. Now working!
  • realize a beta user had included spaces in their path names, so fixed those for him, but need to add clues and validation...

Jun15

  • SplashPage color blocks! And tweak wording.
  • redirect no-path to outer-garden
  • redirect empty-page to edit-page (if owner), per PhilJones
    • blerg, this bypasses seeing the Backlinks, should I add them to edit-page?
    • then re-do video

Jun18

  • had paid for my own membership, to reality-test stripe.com flow
  • just got my email warning me my trial is ending, so going to cancel to test that
  • the Manage Billing link just takes me back to /private!
    • derp, I forgot which account I had activated. Once I logged in with the right account (id=70), the cancel worked fine.

Jun19

  • but looking at that now-canceled account, it's still active+permitted. Is that because cancels keep to alive until the end of the billing period?
  • no fresh events in the table
  • blerg my app hasn't been writing its file log... (it started again once I restarted) so can't browse log to see what other events Stripe sent me....

Edited:    |       |    Search Twitter for discussion

Made with flux.garden