A great use for reddit is the ability to search posts and opinions about any niche topic. Will that be possible with Lemmy as it grows? Will I be able to Google “instant rice Lemmy” and get a comprehensive tier list of each brand?

I imagine search engines will have trouble with all the different instances(?). EDIT: Especially with instances that don’t have Lemmy in their name, I don’t think search engines would return them for Lemmy searches?

  • marsara9@lemmy.world
    73·
    1 year ago

    So I’ve been working on a solution for this.

    As I see it Google and others are going to have a hard if not impossible time to incorporate the fediverse, and the fact that the same content can exist on multiple servers.

    So I’m working on a search engine specifically build, for Lemmy at least. Where it’ll take you to whatever your preferred instance is when tapping on a search result.

    I hope to have a MVP up and running in a few more days.

    • mookulator@lemmy.worldEnglish
      12·
      1 year ago

      Can’t emphasize enough how important this is for the growth of Lemmy. Many people I know only access Reddit through google searches.

      • marsara9@lemmy.worldEnglish
        10·
        1 year ago

        Yep and I’m one of them. Go look me up on Reddit and I think I have maybe 20 posts over the 14+ years I was on the site. …joined Lemmy and immediately got frustrated that I couldn’t find anything. So I figured I take a crack at it. Especially since I couldn’t see how Google would ever be able to link me to my instance. Let alone make it easy to search the entire fediverse without having to write out every possible site, with new ones popping up every day.

        • Aki@lemmy.worldEnglish
          1·
          1 year ago

          I wonder if it’s possible to have a sophisticated search engine similar to Google’s, with BERT and kNN or vice versa. It would be the closest thing to Google search but specifically for Lemmy posts.

    • QuinicV@lemmy.worldOP
      3·
      1 year ago

      Interesting. I hadn’t even thought about how the fact that instance1.[post] and instance2.[post@instance1] is essentially the same thing and how search engines would handle it. Interested in what you come up with!

      • marsara9@lemmy.world
        5·
        1 year ago

        Thanks. If you do some digging you can find the project on GitHub but note that it’s a work in progress still. The UI is lacking and it’s rough around the edges but it’s “working”. And I still need to do some optimizations on the crawler itself, etc…

        It’s also going to be completely self-hostable just like Lemmy, etc…

    • silly goose meekah@lemmy.world
      1·
      1 year ago

      IDK, isn’t it the same for reddit? It also encourages crossposting, so the same content is on there several times. Maybe I don’t understand the fediverse well enough yet, so please correct me if I’m wrong.

      • 1st@kbin.social
        1·
        1 year ago

        On reddit you may have the same post twice but the comments will be different. On Lemmy, you have the same post on every federated lemmy with the same comments on all of them. With the way google handles websites right now, if they started including Lemmy instances in their web, it end up having hundreds of the exact same result each hosted on a different lemmy instance.

        Edit for clarity: All lemmy sites share their data with each other unless they explicitly stop doing so (defederating). This is why I can respond to your comment even though I’m on kbin.social and you’re on lemmy.world

    • static@kbin.social
      1·
      1 year ago

      The mastodon crowd was verry anti on search engines, and killed projects like this.
      But yea, do it! I think the lemmy/kbin crowd would mostly like it

    • Doll_Tow_Jet-ski@kbin.social
      1·
      1 year ago

      That is great. Thanks for the initiative. Have you considered contacting the people at DuckDuckGo so that that search engine can access Lemmy/Kbin content?