Hey!!!!!

That is my website! To be fair, the hard part is hard to keep a personal website regularly updated without making people think it's abandoned. I don't have a regular post cadence. So it looks like I don't touch the website at all for months. But I regularly update my posts and other sections event if there isn't any new posts.

I also wrote something similar to OP - https://www.unsungnovelty.org/posts/10/2024/life-of-a-blog-b...

And I'd like to also mention https://marginalia-search.com/ which is a small OSS search engine I have been using more and more theese days. I find it great to find IndieWeb / Small Web content.

Anyone curious this is the same for Linux, except use xdg-open like this:

  $ xdg-open $(pages)

This is so lovely. Just adopted it for arch. And set it up, so that I can just type `indy n` (with "n" being any number) and it opens n pages in my browser.

Thanks for sharing.

These curated discovery services require RSS and Atom feeds. My site doesn't even have those. Looks like I'm too small for the small web.

Caveat that Kagi gates that repo such that it doesn't allow self-submissions so you're only going to see a chunk of websites that other people have submitted that also know about the Kagi repo.

That top one only updates once a year. Not saying that as a criticism, just how lucky he was to update recently enough to end up in this top comment

That's a lovely bit of automation.

Multiple layers of curation works really well. Specifically, using HN as a curation layer for kagi's small web list. I implemented this on https://hcker.news. People who have small web blogs should post them on HN, a lot of people follow that list!

This is great. There's something refreshing about discovering websites by accident instead of through an algorithm optimizing for engagement. I've been spending more time on personal blogs lately and the writing quality is consistently better than anything on Medium or Substack probably because nobody's writing for SEO.

A beautiful trend that has been going for 30 years ;-)

One of the happiest moments of my childhood (I'm exagerating) was when my button was placed in that website that I loved to visit everyday. It was one of the best validations I ever received :)

my main problem with such links is... how often do you update them? how often do you check those websites to see that they're still active?

I remember going through all the blogs linked on terry tao's blog - out of like 50 there were only 8-ish still alive :(

gotta be using pixel fonts for those, they been around for 25 years, actually readable at 8px lol

but why link to apple.com and vercel.com?

One of the many things I disagree with Scott Alexander on is that to me, frequent blog updates signal poor quality not excellent writing. Its hard to come up with an independent, evidence-based opinion on something worth sharing every week, but easy to post about what you read lots of angry or scary posts about. People who post a lot also tend to have trouble finding useful things to do in their offline life. It is very unusual that he managed to be both a psychiatrist and a prolific blogger and he quit the psychiatry job before he had children or other care responsibilities.

I'm with you. Also, sometimes I'm specifically looking for some dusty old site that has long been forgotten about. Maybe I'm trying to find something I remember from ages ago. Or maybe I'm trying to deeply research something.

There's a lot more to fixing search than prioritizing recency. In fact, I think recency bias sometimes makes search worse.

To clarify criteria is less than 2 years since last blog post.

also kagi exclude non-English sites. Sad for mixed language blogs like mine.

Suspect there's a long tail/iceberg you still haven't captured (source: you haven't found me yet and I'm not hiding, I'm just not chasing SEO).

> I'd like to think that we have captured most of (english speaking) personal blogs

I think that's naive.

But maybe thats just because my blog wasn't on the list :)

What methods are you using to find them? I notice my own doesn't appear, although it does show up well under some (very niche) Google search terms. I suspect there's the potential for an order of magnitude more sites than have been found.

Does this use frames or iframe? https://kagi.com/smallweb

I would expect a raw link in the top bar to the page shown, to be able to bookmark it etc.

Does this concept of "personal blog" include people periodically sharing, say, random knowledge on technical topics? Or is it specifically people writing about their day-to-day lives?

How would I check if my site is included?

I'm noticing sites that break the rules. I report (flag) them, is that useful or should I just PR to remove them?

It doesn't have mine, because no rss

[dead]

You want the government to fund the small web?

Are you asking why public libraries exist?

My point also: I rather see the existing web transformed than being part of some obscure circles that not many people care about.

The tag cloud part may be a challenge. Web feeds don't always tag their content.

I have a blog filter that does something similar (https://alexsci.com/rss-blogroll-network/discover/), but the UI I ended up with isn't great and too many things are uncategorized.

I also think equating good = "no monetization" is exactly how we've ended up in a situation where everything is controlled by a few giant mega corps, hordes of MBAs, and unethical ad networks.

We should want indie developers, writers, etc to make money so that the only game in town doesn't end up being those who didn't care about being ethical. </rant>

Yep. People have very short memories. I remember that ethical ad network in the late 2000s that all the cool tech bloggers would use.

I would put it all on cookies. No third party cookies (at all) - good. JS and CSS and even autoplay video is fine as long as there are no third party cookies.

That would make the Small Web bigger but it would get to the main point. I'd be fine with a site like the New Yorker that has more bells and whistles be included as long as I could experience it without a tracked ad from DoubleClick.

Right now any serious outfit simply cannot be included in the Small Web but we really need companies there.

Yeah, CSS is Turing Complete: https://lyra.horse/x86css

Most of the problem with ads isn't even the ads these days, but the bloat. Static image ads would be a huge improvement.

You need to go more tin-foil-hat

Its not just JavaScript, it's cookies, it's "auto loading" resources (e.g. 1x1 pixels with per-request unique URLs), it's third-party http requests to other domains (which might art cookies too).

I think the XKCD comic about encryption-vs-wrench has never been more apt for Gemini the protocol...

Thanks for shilling.

Regarding the financials, even though the second nlnet grant runs out in a few weeks, I've got enough of a war chest to work full time probably a good bit into 2029 (modulo additional inflation shocks). The operational bit is self-funding now, and it's relatively low maintenance, so if worse comes to worst I'll have to get a job (if jobs still exist in 2029, otherwise I guess I'll live in the shameful cardboard box of those who were NGMI ;-).

I think that's a cool project, though I found the results to be less relevant than Google.

> Google that now optimizes their algorithm for monetization and not usefulness.

I don't think they do that. Instead, "usefulness" is mostly synonymous with commercial intent: searching for <x> often means "I want to buy <x>".

Even for non-commercial queries, I think the sad reality is that most people subconsciously prefer LLM-generated or content-farmed stuff too. It looks more professional, has nice images (never mind that they're stock photos or AI-generated), etc. Your average student looking for an explanation of why the sky is blue is more interested in a TikTok-style short than some white-on-black or black-on-gray webpage that gives them 1990s vibes.

TL;DR: I think that Google gives the average person exactly the results they want. It might be not what a small minority on HN wants.

Big thing that made encryption required is arguably that ISPs started injecting crap into webpages.

Governments can still track you with little issue since SNI is unencrypted. It's also very likely that Cloudflare and the like are sharing what they see as they MITM 80% of your connections.

> But what I think is an even better solution is to do it at the content level: sign the content, like a GPG signature

How would this work in reality? With the current state of browsers this is not possible because the ISP can still insert their content into the page and the browser will still load it with the modified content that does not match the signature. Nothing forces the GPG signature verification with current tech.

If you mean that browsers need to be updated to verify GPG signature, I'm not sure how realistic that is. Browsers cannot verify the GPG signature and vouch for it until you solve the problem of key revocation and key expiry. If you try to solve key revocation and key expiry, you are back to the same problems that certificates have.

I've been thinking the same thing for years -- thank you for saying it. I agree completely.

Another pro is that no encryption means super low power microcontrollers and retrocomputers can browse freely. The system req's go down by orders of magnitude. I think enforcing TLS in the Gemini protocol was a huge mistake; there are so many retrocomputing enthusiasts that would love to browse Geminispace on their Amigas and 486s -- it might actually have been a significant part of the userbase -- but they're locked out because their CPUs simply cannot reasonably handle modern TLS.

I have noticed that when I encounter an HTTP-only web site, I know I am in for a pleasant, calm, well-curated experience, and I mean that without a hint of irony.

I don't have a lot to say about the technical discussion here, other than "TLS null cipher could be fine but also a lot more infrastructure than desirable", which could subvert your intent here.

Maybe we should normalise TOR usage before it becomes a surefire signal to the FBI to raid one's home.

> It also helps technically. If you want to make a tiny web server, like on a microcontroller, encryption is the hardest part.

> Two problems remain with the lack of encryption, first is authenticity. Anyone can man-in-the-middle and change the web page, TLS prevents that. But what I think is an even better solution is to do it at the content level: sign the content, like a GPG signature, not the server, this way you can guarantee the authenticity of the content, no matter where you are getting it from.

If your microcontroller can't do TLS then it probably won't do GPG either. But you can still serve HTTP content on port 80 if you need to support plaintext. I believe a lot of package distribution is still over HTTP.

Edit: Sorry, missed the web server part somehow and was thinking of a microcontroller based client.

> In addition, TLS comes with expiring certificates, requiring regular maintenance, you can't just have your server and leave it alone for years, still working. It can also bring back simple caching proxies, great for poor connectivity.

Yeah, TLS and DNS are the two of the biggest hurdles to a completely distributed Internet. Of course you go down that road and you get IPFS, which sounds cool to me, but doesn't seem to have ever taken off.

Anyone between you and the server can change the content of the page on unencrypted connections. I would love to live in a world where encryption is unnecessary, but unfortunately that world does not exist right now.

The internet has become more hostile since its early days, in multiple ways and one of those ways is that networks spy on you more. They used to inject ads into the content, but that stopped being profitable when the majority went to HTTPS. If they could do it again on a large scale they would. The NSA is also saving all the data they can. It's important to hide the content, now we have first-world countries using information about what you read to make life-or-death decisions. Encryption should now be seen as necessary.

> If you want to make a tiny web server, like on a microcontroller, encryption is the hardest part

Even an esp32 can (just) handle TLS. Given relatively modern designs, you end up on remarkably small chips before TLS is a real blocker

I think a simpler argument would be that small web is not a good fit if your content is sensitive in the place you are publishing from. It’s meant for public publishing. If you need encryption, use a different distribution mechanism.

Your entire comment is paragraphs of grasping at straws.

> The other thing is the usual argument about oppressive governments, etc... Well, if want to protect yourself, TLS won't save you, you will be given away by your IP address, they may not see exactly what you are looking at, but the simple fact you are connecting to a server containing sensitive data may be evidence enough. Protecting your identity is what networks like TOR are for, and you can hide a plain text server behind the TOR network, which would act as the privacy layer.

A huge ‘citation needed’ for this whole paragraph. Just admit that you don’t care about this use case and move on. Don’t present a contrived and completely justified hypothetical where oppressive governments behave exactly in a way that happens to mean that there’s only room for the technologies that you personally are into.

You’ve completely departed from reality. It’s not 2004 anymore.

I think the small web ought to just serve everything over Tor (onion sites). No domain needed, no worries about surveillance. You can put a gateway in front of your minimalist web server or run the gateway on an OpenWRT router. This also helps the network by generating cover traffic and encouraging civilian use.

TLS client certs are quite a nice approach to identity though.

>Removing encryption means that you can't reasonably do financial transactions, accounts and access restriction, exchange of private information, etc... You only share what you want to share publicly, with no restrictions. It seriously limits commercial potential which is the point.

People will still do financial transactions on an unencrypted web because the utility outweighs the risk. Removing encryption just guarantees the risk is high.

> I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking.

I think it should allow but not require encryption.

> Removing encryption means that you can't reasonably do financial transactions, accounts and access restriction, exchange of private information, etc... You only share what you want to share publicly, with no restrictions. It seriously limits commercial potential which is the point.

Note that the article linked to says "the Gemini protocol is so limited that it’s almost incapable of commercial exploitation", even though Gemini does use TLS. (Also, accounts and access restriction can sometimes be used with noncommercial stuff as well; they are not only commercial.)

> It also helps technically. If you want to make a tiny web server, like on a microcontroller, encryption is the hardest part.

This is one of the reasons I think it should not be required. (Neither the client side nor server side should require it. Both should allow it if they can, but if one or both sides cannot (or does not want to) implement encryption for whatever reason, then it should not be required.)

> Anyone can man-in-the-middle and change the web page, TLS prevents that. But what I think is an even better solution is to do it at the content level: sign the content, like a GPG signature

Using TLS only prevents spies (except Cloudflare) from seeing or altering the data, and does not prevent the server operator from doing so (or from reassigned domain names, if you are using the standard certificate authorities for WWW; especially if you are using cookies for authentication rather than client certificates which would avoid that issue (but the other issues would not entirely be avoided)).

Cryptographic signatures of the files is helpful, especially for static files, and would help even if the files are mirrored, so it does have benefits. However, these are different benefits than those of using TLS.

In other cases, if you already know what the file is and it is not changing, then using a cryptographic hash will help, and a signature might not be needed (although you might have that too); the hash can also be used to identify the file so that you do not necessarily need to access it from one specific server if it is also available elsewhere.

> Well, if want to protect yourself, TLS won't save you, you will be given away by your IP address, they may not see exactly what you are looking at, but the simple fact you are connecting to a server containing sensitive data may be evidence enough.

There is also SNI. Depending on the specific server implementation, using false SNI might or might not work, but even if it does, the server might not provide a certificate with correct data in that case (my document of Scorpion protocol mentions this possibility, and suggestions of what to do about it).

The fediverse is also generally experienced as a small web, where it comes to mindset. Though that is not always to the liking or preference of those expecting to find alternatives to big church social media platforms.

Feeding llms you mean

I mainly use Kagi Small Web as a starting point of my day, with my morning coffee. Especially now when categories are added, always find something worth reading. The size here does not present a problem as I would usually browse 20-30 sites this way.

> The question is how do you take it to a million?

Do you need to take it to a million in the same place? Is that still "small"?

Why not have 2000 hand curated directories instead?

My approach operates under the assumption that good, non-commercial webpages will be similar to other good webpages. Slop, SEO spam, and affiliate content will resemble other such content.

So a similarity-based graph/network of webpages should cluster good with good, bad with bad. That is what I've seen so far, anyway.

With that, you just need to enter the graph in the right place, something that is fairly trivial.

Reminds me of https://notes.pault.ag/tpl/

Parts of the "dark web" are built somewhat like this

However if it's supposed to continue being an indie protocol and fly under the radar, it's perfect

There's an element of nostalgia, certainly but it's also a reaction to the overwhelmingly commercial web. Why not build something instead of scrolling through brief videos interspersed with more and more ads that follow you everywhere?

Large companies have helped build the web but they've done at least as much, if not more, to help kill it.

It's about capturing the noncommerciality, not the experimentation. Most of the small web sites are just blogs, a solved problem by now, but there's interesting content in many of them.

Which is exactly the point of Gemini.

The comment belongs to this thread https://news.ycombinator.com/item?id=47401809

The experience of the internet would be so much more interesting if the search engines unearthed rare blogs or writing from small creators and bloggers that thought things through or shared original ideas.

It did seem we had that for a while and now everything funnels back to a handful of big platforms.

Maybe as AI swallows the data of the entire web, it would start to look for these small sites, small creators, and rare personal content to keep itself interesting and we'll see more of them?

>I dint think it's malicious or even intentional...

it's indirectly intentional in that Google isn't wringing it's hands trying to destroy tiny blogs but they (Google) have deliberately chosen to ignore anything that doesn't play the SEO game, whatever the driver of that game is.

> I dint think it's malicious or even intentional--but at some point they stopped directing traffic to small websites.

Small websites have small dollars?

Bro are you kidding? The whole point of small web is to avoid cooperations, AI and governments.