WikiGraph With FreeLinks
urlfield for the (Camel-Case for-me) page-bit of the URL (don't include the path part)
namelower-case, so effectively get case-insensitive matching
- (do I also want a clean separated version of this value for a
titletag that would be nicely word-searchable, to make more general SearchEngine? Nah, can always just do regex.)
- now when widget calls the function
- have to convert Referer to SmashedTogetherWords format if it's not already
- then make sure using the url value in the return, not the name
- ask key engine-runners: Ward Cunningham and Mike Caulfield
- do their engines have an AllPages page to scrape?
- do they have a directory of spaces? naming convention per space (with uniqueness)?
- has all-spaces FrontPage which you can crawl through Prev pages
- smells like it doesn't enforce name, but can generate from hostname+domain (drop the cc)
- scrape some data
- build (update) system
- start using on WebSeitz
- then let them call TwinPages widget
- then SearchEngine
- then copy from SearchEngine to local space
- see MikeCaulfield BootstrappingLibrary/ColdStart post
- store copied-from URL?
- allow copy to different name?
- hmm does my WikiEngine need Edit button for guests, so they can copy?
Mar08'2016: write parser for (1st page of) links
Mar09: create old db in local postgres, import old records. (Was in SQLite before.)
Mar09: tweak Wikity parser code to make pages.url lower-case
Mar09: scrape all wikity pages, parse (1766 urls, 75 spaces)
Mar09: get old/current wikigraph db/app working
- WikiGraph/WebPy won't talk to psql, realize none of my old WebPy apps ever used it, always used SQLite
- Launch WikiGraph using SQLite, tweak WikiWeb footer to call it. It works!
- change db structure, fill past fields, import couple records
- adjust code to use new db
Mar10: map Referer to space to support wikitys and not link to self
Mar11: adjust import code for extra field, and for spaces table
Mar11: native search form - WikiSearch
- ugh bug when spaces.page_pattern includes a path, like for me! Hacked for me, so working. But need to fix.
Mar12: tell MikeCaulfield
Mar12: add custom SearchEngine! https://cse.google.com/cse/all
Mar14: add Phil Jones SFW space after he emails me list. Need to fix bug from Mar12 for him to use the widget too
Later: handle SFW sites
- scrape Google? or use Ward's scraper code?
Edited: | Tweet this!