On search engines and finding things

Search has been top-of-mind for me lately. I’m definitely an information addict - there’s nothing I love more than finding a tasty morsel of knowledge to spark an idea or inspire my creativity. It can be anything; a new recipe, a gadget to hack on, memes, open source software, a piece of history, music.

Finding the right knowledge to start my engine is an art, not a science. Everything is everywhere. Sure, we have traditional search engines like Google, Bing, and Kagi, but these are limited. They are laser-focused on answering questions, not inspiring creativity.

I want a search engine which can show me to my next rabbit hole.

I think this is why I’m so drawn to the small web. Real people, like me, who have crazy ideas and opinions and strange skillsets. People motivated by a desire to just do random shit because it interests them for some reason. These are the kinds of people hacking, designing, and making things, then blogging about them despite not really having an audience to cheer them on. Search engines, link directories, and blogrolls for finding content from creators of this ilk already exist and work quite well, but they’re scattered across the web and come in many different shapes and sizes.

This is why I’m building freectl, a tool for finding cool stuff. It’s self-hostable, offline (besides retrieval and updates), and fast. Dual CLI and web UI.

For now it can find links in GitHub repositories, but I plan to add support for:

  • Reddit megathreads
  • HackerNews top 5000 blogs
  • Blogroll.org
  • OPML import
  • Bookmarks import (HTML export, Hoarder, Linkwarden)
  • Personal knowledge stores (Obsidian vaults/markdown documents)
  • Open directories
  • Other link dumps/datasets

I’d like to avoid web crawling, so to begin with I’ll be relying on existing datasets. Searching uses fuzzy search but I’d like to try indexing and full-text search with bleve too. Indexing opens doors to other features like AI-enhanced discovery, so I’m keen to get that working over the next few weeks.

Like with most things I spend time doing on my computer, my engine will eventually stall and I’ll tumble into the next rabbit hole. But for now at least, finding rabbit holes is my rabbit hole.