• jbk
    link
    fedilink
    arrow-up
    32
    ·
    3 months ago

    I thought major LLMs ignored robots.txt