But you’re ignoring the entire enterprise side of things. MS Recall + pervasive data mining and ad injections are things that the vast majority of IT departments are going to refuse to sign off on. These technologies meaningfully and fundamentally undermine organizational and system security, up to and including potential inadvertent exposure of cryptographic secrets, which the modern internet is basically built on top of.
Sure, consumers are likely going to acquiesce out of either laziness or ignorance. But IT orgs aren’t going to simply sign off on this - particularly if they’re operating in an industry where InfoSec really matters (basically, any regulated industry like medical, biotech, or aerospace).
There is a huge corporate insensitive that everyone is not realizing here. By screen recording + OCR, there is a possibility to start using this data to replace some labor intensive, but simple tasks of operating a business. If you can create RPA+ML+LLM that can rerun repetitive tasks, you have holy grail on your hands. I think this is one of the big reason why M$ is pushing this.
I assume to be down voted to oblivion, but I do business automation and integration for living, and at the same time I am scared and excited.
Absolutely. Corporations - at least, shitty ones (most of them) - are absolutely salivating at using this. They want to be able to see and easily summarize eeeeeeverything you’re doing.
Some are absolutely already using a form of this. It’s not a hypothetical - this is currently happening and many want way way more.
Lmao do you have any idea how quickly that’s going to go off the rails? They’re going to get into a hallucination feedback loop, which will destroy the integrity of their systems and processes, and they’ll richly deserve it.
At any rate, most highly-effective technical teams have already automated the shit out of all their rote operations without using ML.
Automation suites exist and they are very much tuned to the individual apps. It seems giving ML an OCR readout of a page is not enough for it to know what it should do (accurately). We have had a training set for “booking flights on a browser” for about 6 years now and no one has figured out how to have it disrupt automated testing: https://miniwob.farama.org/
I was thinking about this, but I don’t know what the plan us for annotating new flows with descriptions of the actions. There’s no point in learning how to send an email or open a webpage, that’s already easy. The value is in a database of uncommon interactions, but it’s only valuable if there is a description to train on.
Perhaps, but at this point, the only ones who actually know the endgame strategy are product people at MS, and they’re almost certainly bound by NDAs on that topic.
Unfortunately most large organizations are running on enterprise releases that only lay down minimal software. Plus IT depts have heavily maintained images that immediately shuts off anything that sneaks in. Help desk is just going to disable the feature before slapping the company background image and VPN on it and giving it to standard users. They will make a ton of money in the short term and EOL the operating system when it’s no longer profitable and Linux is the default (decades from now). AOL is still out there
But you’re ignoring the entire enterprise side of things. MS Recall + pervasive data mining and ad injections are things that the vast majority of IT departments are going to refuse to sign off on. These technologies meaningfully and fundamentally undermine organizational and system security, up to and including potential inadvertent exposure of cryptographic secrets, which the modern internet is basically built on top of.
Sure, consumers are likely going to acquiesce out of either laziness or ignorance. But IT orgs aren’t going to simply sign off on this - particularly if they’re operating in an industry where InfoSec really matters (basically, any regulated industry like medical, biotech, or aerospace).
There is a huge corporate insensitive that everyone is not realizing here. By screen recording + OCR, there is a possibility to start using this data to replace some labor intensive, but simple tasks of operating a business. If you can create RPA+ML+LLM that can rerun repetitive tasks, you have holy grail on your hands. I think this is one of the big reason why M$ is pushing this.
I assume to be down voted to oblivion, but I do business automation and integration for living, and at the same time I am scared and excited.
Absolutely. Corporations - at least, shitty ones (most of them) - are absolutely salivating at using this. They want to be able to see and easily summarize eeeeeeverything you’re doing.
Some are absolutely already using a form of this. It’s not a hypothetical - this is currently happening and many want way way more.
Lmao do you have any idea how quickly that’s going to go off the rails? They’re going to get into a hallucination feedback loop, which will destroy the integrity of their systems and processes, and they’ll richly deserve it.
At any rate, most highly-effective technical teams have already automated the shit out of all their rote operations without using ML.
Automation suites exist and they are very much tuned to the individual apps. It seems giving ML an OCR readout of a page is not enough for it to know what it should do (accurately). We have had a training set for “booking flights on a browser” for about 6 years now and no one has figured out how to have it disrupt automated testing: https://miniwob.farama.org/
I was thinking about this, but I don’t know what the plan us for annotating new flows with descriptions of the actions. There’s no point in learning how to send an email or open a webpage, that’s already easy. The value is in a database of uncommon interactions, but it’s only valuable if there is a description to train on.
They’ll just keep it shut off. It’s not a requirement.
Will they not have licenses with all of this shit stripped out? Maybe another way to force ITs to pay for proper licensing and stuff too 🧐
Perhaps, but at this point, the only ones who actually know the endgame strategy are product people at MS, and they’re almost certainly bound by NDAs on that topic.
Unfortunately most large organizations are running on enterprise releases that only lay down minimal software. Plus IT depts have heavily maintained images that immediately shuts off anything that sneaks in. Help desk is just going to disable the feature before slapping the company background image and VPN on it and giving it to standard users. They will make a ton of money in the short term and EOL the operating system when it’s no longer profitable and Linux is the default (decades from now). AOL is still out there
They didn’t have it set up to be easily disable-able, as far as I understand it.
Are they? Some IT departments are going to love the invasive nature of it.
Then those IT departments should be blackballed from the industry, because the nature of that invasive surveillance is WILDLY insecure.