(2020-10-02) Setting Up Python On New MacBookPro

Setting Up Python On New MacBookPro - from (2020-09-23) Time To Replace MacBookPro

Goals

• Python 2 and 3
• OpenSSL current
• clean up some of my project/directory structures
  • hierarchy based on WebAppFramework; peer directory for non-WebApp pieces - root at Documents/code/py2 (not /djcode/) - will do pyenv local 2.7.18
    • flask
      • wikiweb
      • wilson (coachbot webapp)
    • webpy
      • mf (moneyflux, familyfinancialplanning)
      • wikigraph - *special case includes lots of stuff I might move)
      • mss (MySpaceShip)
      • tddwp (Test Driven Development With Python)
    • other(?) (or just pop these up a level? No, want single virtualenv shared across them)
      • wilson (coachbot command-line)
      • autopost
      • insta_repost (move up out of wikigraph)
      • TwitterBots
• hmm do I want to plan on Anaconda, anticipating stepping up ML work in CoachBot? Nah let's keep it simple.

Mac OS-X 10.15.7 - Catalina

PyEnv vs VirtualEnv vs Ana/conda vs ???

Oct03'2020

• going with pyenv, following these instrux
  • homebrew
  • pyenv
  • pyenv -> profile
  • python --version -> Python 2.7.16
  • xcode-select --install
  • brew install openssl readline sqlite3 xz zlib
  • pyenv install 2.7.18
  • pyenv install 3.8.5 (tried 3.8.6 but was just released a week ago, and pyenv doesn't have it)
• hmm what Flask version am I running?
  • Linode: 0.11.1 (on python 2.7.6)
  • old MacBookPro: no Flask! I suspect my thrashing attempts to deal with OpenSSL a couple months ago wiped it out or something.... Onward...
  • so what should I install on this new laptop?
    • Jump to latest, get my code working, then update Linode to match?
    • Match Linode? Going to take this approach, seems less risky (if I can install the bits I want)
• mkdir code/py2 (not quite), go into dir, pyenv local 2.7.18
• Flask docs suggest virtualenv, so do that
  • need pip: download get-pip.py, do python get-pip.py
    • WARNING: The scripts pip, pip2 and pip2.7 are installed in '/Users/billseitz/Library/Python/2.7/bin' which is not on PATH.
  • python --version → Python 2.7.16 wtf?!? Why is it using the system Python?!? Ah, close, start new Terminal window, now it works (I had done a command from earlier instructions that should have used the updated .zprofile but maybe that didn't work
  • do python get-pip.py again
  • pip install virtualenv
  • mkdir flask, cd flask, virtualenv venv
  • source venv/bin/activate This is key thing to run every time I'm in this project.
  • pip install Flask==0.11.1 (per this) - seems like it worked
• copy the project directories over under /flask/ from old laptop
• python hello.py → Running on http://127.0.0.1:5000/ - hit the URL, it works!
• go into /wilson/ project directory, python wilson.py get errors about missing sqlalchemy (pretty sure I don't use db in that app, but probably have stuff installed anyway) (not going to worry about that for now)
• switch to /wikiweb/ directory, python wikiweb.py get same sqlalchemy alert
  • any requirements.txt file to give any clues - nope
  • error also noted flask.ext.sqlalchemy is deprecated - so shall I install the newer flask_sqlalchemy and hope I can move forward from there? OK let's try that.
  • pip install -U Flask-SQLAlchemy → good Successfully installed Flask-SQLAlchemy-2.4.4 SQLAlchemy-1.3.19

Oct04 continuing

• pip install psycopg2 → Error: pg_config executable not found
  • launch my Postgres.app → empty data directory → press Initialize button, it creates folder
  • sudo find / -name "pg_config" -print → /Applications/Postgres.app/Contents/Versions/13/bin/pg_config
  • add PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH" to bottom of .zprofile
  • pip install psycopg2 successful
• python wikiweb.py → issue with from werkzeug.contrib.atom import AtomFeed, No module named contrib.atom
• at some point werkzeug dropped atom
• what version of werkzeug do I have?
• find → /code/py2/flask/venv/lib/python2.7/site-packages/werkzeug, cat __init__.py → __version__ = "1.0.1"
• per prev link, pip install Werkzeug==0.16.1
• python wikiweb.py → success!
• hit the URL, get empty page, plus error about database does not exist. As expected.
• back to python wilson.py which doesn't use db. ImportError: No module named CommonMark
  • pip install commonmark → Successfully installed commonmark-0.9.1 future-0.18.2
  • run wilson.py again, same error
  • hmm going to let this slide for now, it may come up in wikiweb since I use CommonMark there, too.
• migrate data, use pg_dumpall
• python wilson.py no empty-db error! but get the CommonMark error!
  • grr I see it in /site-packages/; I've started new terminal window, gone back into venv, etc.
  • change my code's references from CommonMark to commonmark
• launch again - ImportError: No module named pyparsing
  • pip install pyparsing
  • FrontPage renders!
  • single-page renders!
• do login - when submit id/password it's just sitting there... but if I go back and reload page I was on, I can see I'm now logged in. And can switch over to private pages. And can edit/save.
• make CommonMark changes to wilson, can now run that fine.

Oct04: on to webpy - it's not dead! The latest 0.61 release supports Python >= 3.5. Version 0.51 is the last release with Python 2.7 support.

• mkdir /code/webpy, cd webpy, virtualenv venv, bin/activate
• old mac was using 0.37
• pip install webpy==0.37 → ERROR: Could not find a version that satisfies the requirement webpy==0.37
  • hmm see it listed here
  • ah it's pip install web.py==0.37 → success
• copy over old directories of webpy code - except wikigraph
• cd mf, python code.py → no error
• hit http://0.0.0.0:8080/, get splash page
• hit some other pre-login pages, even those pulling from db work fine!
• I don't remember my login, so I'm not even going to bother.
• same for mss
• hah realize that both of these apps uses MySql for their local versions
  • look up my admin login info
• hit mf login - submit id/pw → No module named cryptacular.bcrypt
  • pip install cryptacular==1.4.1 (that's the version on old laptop)
  • relaunch server, hit page, get private db-driven page fine
• launch mss, hit/submit login, get db-driven private page
  • hmm clicked into a detail page, got No module named markdown
  • wasn't markdown bundled? Maybe not quite.
    • found earlier snapshot of page pointing to http://www.freewisdom.org/projects/python-markdown/markdown.py which doesn't exist.
    • old laptop had version 2.2.0
  • pip install markdown==2.2.0 successful - and now that page loads fine.

Oct05: wikigraph -

• copied over "core" files from old laptop
• launch server: page comes up, but D3 graph in the middle doesn't show anything; decide to try the key widgets first
• tweak local wikiweb to call local wikigraph, hit wikiweb page
• error, in referer_parse function - I have some cases already to handle local URLs etc, but
  • it's coming through now as 0.0.0.0 instead of localhost
  • check both cases → now works!

Oct06: look at side-bits that were buried inside wikigraph

• create /other/
• organize wikigraph muck into clean sub-folders, copy over; also copy over the other non-server code directories
• virtualenv venv, activate
• going to try Instapaper-related bits before TwitterBots (but copied the twitterbot folder over)

Oct06 InstaPaper

• (tweak directories used in code for pipeline files)
• python insta_repost.py → ImportError: No module named oauth2
• old laptop's oauth2 is v 1.9.0.post1, which still has a reference page
  • then there's python-oauth2 1.1.1 whose page says If you are looking for a OAuth 2.0 library to integrate into your Python application, I recommend oauthlib.
  • go for backward compatibility: pip install oauth2
• python insta_repost.py - success! Edit a highlight file....
• python autopost.py → ImportError: No module named requests
  • old laptop has v 2.18.4
  • pip install requests==2.18.4 → success
• python autopost.py → success!

Oct06 TwitterBot

• cd sycorax, python nudge.py → ImportError: No module named twitter
  • old laptop has v 1.3.1
  • pip install twitter==1.3.1 → success
• python nudge.py → AttributeError: 'module' object has no attribute 'Api' (from api = twitter.Api())
  • ah, it's python-twitter I need (reading from TwitterBot page); pip install python-twitter==1.3.1
• python nudge.py → same error grrr
  • pip uninstall twitter
• python nudge.py → twitter.TwitterError: [{u'message': u'Invalid or expired token.', u'code': 89}]
• python get_access_token.py → ImportError: No module named oauth.oauth see TwitterBot notes.
• or should I try jumping to BotFriend?

Nope, I realize TwitterBot stuff isn't actually a priority, esp since it's already been broken for a year. So just going to let this alone and move forward on the rest of the laptop-migration tasks.

(Late Oct07 - did WikiGraph update which is further confirmation of some things working, though that's pretty decoupled from servers/etc so it doesn't say much...)

Nov14 - laptop rebooted the other day. Now insta_repost isn't working - giving me ImportError: No module named oauth2 despite prior setup Oct06. Do pip install oauth2 again, now can run code.

(2021-02-07) Migrating to Python3