evaluate platform for how-to successor

[antidipyramid]

Background

how-to has become unwieldly to update and much of the documentation is out of date. We will explore using a different platform than GitHub markdown to house DataMade's documentation and development practices.

A new how-to should:

• be easy to update with minimal friction for edits if desired
• have good search and discoverability
• give us control over the platform to ensure our key features are not enshitified
• the tool should default to private, with an option to publish individual pages or entries

Continuation of #409

[hancush]

Is it important for our documentation to be public?

[yshng]

@hancush There was some back in forth on this - I'm generally a subscriber to the idea that you should start in public, and then go to private when the need arises. As part of creating the kind of internet I'd like to exist. I think others instinctively leaned private but were open to public-first.

However, I would completely flip-flip on this if having things be private by default somehow meant that there would be meaningfully less friction for the process of adding/editing documentation. Ease of access/update means that the docs will actually get used and be useful and that's the main priority imo

[yshng]

Per issue #417, we're looking for a new platform for our internal documentation.

Based on our team discussion at R&D meeting (sans @hancush, who weighed in later),
it should:

• be easy to update with minimal friction for edits if desired
• have good search and discoverability
• give us control over the platform to ensure our key features are not
  enshitified

We may keep the platform public, private, or a mix of the two, TBD.

The contenders

Notes style

Obsidian

Some of the team already use Obsidian (Derek, Hannah).

Obsidian itself is free as in beer, not free as in freedom. Team use could use a vault hosted on Google Drive (or presumably a git repo?) rather than purchasing their Sync service.

• If we wanted to publish docs, there are numerous solutions that people have come up with like this person's workflow for publishing via GH, Jekyll, and Netlify https://stephango.com/vault
• Which is to say, Obsidian could be for internal use and we could still publish things using a different service as below
• A lot of the power and functionality of Obsidian depends on Obsidian flavored Markdown which has several elements not supported in typical Markdown
  • Wikilinks style [[]] links can be turned off, so that's fine
  • Callouts, highlights, and comments are specific to Obsidian (/borrowed from other formats)
  • Any use of additional plugins can add more non-standard elements

A whole app

Bookstack

BookStack is a wiki-type app that we would self-host. It organizes information into "Books".

• email/password auth and supports other options.
• features a WYSIWYG editing interface, including a Markdown editor with live-preview.
• fully searchable
• linkable down to the paragraph level
• UI enhancements like "Recently Updated", "Recently Viewed", "My Most Viewed Favourites"
• Permissions system
• Can't just store/work on straight up .md files https://www.bookstackapp.com/docs/admin/content-storage/

Screenshots
Demo
Source on Codeberg.

It appears to be actively developed, mostly by one guy, and they maintain a Zulip instance for support and updates.

Wiki.js

• Can be public, private, or a mix

• Built in auth or can use third-party auth through Slack, Github, Google, etc.

• Has a community template for deploying on Heroku

• Built-in DB/DB - PostgreSQL search engine, but you can also use a different search backends like Elasticsearch

• Can sync with Github repo for docs storage

• Tags and tags browser

• Has permissions system https://docs.requarks.io/en/groups

• source on github

See their docs site for demo, it runs on wiki.js

• docs site
• Markdown topic with "Edit on GitHub" to see source: https://docs.requarks.io/editors/markdown

Community and Development

• They have a Discord channel, and GitHub Discussion board: https://github.com/requarks/wiki/discussions
• They have a lot of promised features but there's not that much development going on. https://github.com/requarks/wiki/graphs/code-frequency
• All the big companies on their website are kind of lies eg you can deploy via Docker, or they give “special thanks” to GitHub on the sponsors page, but they did not actually receive money from them (or else why would they be at the bottom of the page). V sketch marketing strategy.

Just the docs

MkDocs

• One config file, where you set up nav bar, plugins, etc https://github.com/mkdocs/mkdocs/blob/master/mkdocs.yml
• Uses Lunr for search
• Just make a github repo with .md files and deploy to github pages with a command https://www.mkdocs.org/user-guide/deploying-your-docs/
• Their site, linked above runs on mkdocs
• Themes exist to make it look nicer, e.g. https://asiffer.github.io/mkdocs-shadcn/, which also points to it people using it
• See also Material for Mkdocs A more full featured, currently maintained framework build on top of MkDocs

Community and Development

• GitHub discussion board, https://github.com/mkdocs/mkdocs/discussions
• Basically in the same state since 2024 (which is a working state!)

Quarto

You know about it, we use it for LA Metro.

Docusaurus

• Meta open source project
• Examples: https://docusaurus.io/showcase
• Use MD files, deploy to netlify https://docusaurus.io/docs/deployment#deploying-to-netlify
• Search wants you to use algolia search so you have to sign up and apply for their program https://docusaurus.io/docs/search#using-algolia-docsearch

[hancush]

just adding a note for obsidian: you can host a vault in google drive. i do this with my work vault, so i can access it across devices. would also work for a shared vault.