• jbk
    link
    fedilink
    arrow-up
    32
    ·
    4 months ago

    I thought major LLMs ignored robots.txt