I've had a subscription to ChatGPT since basically Day 1. On one hand I love it, it's really clever, on the other hand.....wow, it can be crap at times.
For me, it's become a secondary form of Google search most of the time - it's pretty good for pointing you towards things that you dont' even know where to start.
On the other hand, it loves a good hallucination and will send you down an utter rabbit hole if you let it. Most of my job is doing "weird" stuff - i.e., developing things that aren't well defined or well solved on the internet, so I have to be really really careful trusting GPT.
I once spent hours trying to get something working that it signposted me towards, only to find out that fundamentally you can't do what it was saying.
It's very very limited, and it terrifies me that people don't understand that
My experience of AI so far is that it's a convincing liar, unless you ask it to do maths, draw hands, or do anything involving anything that anyone has ever been slightly sarcastic about, ever.
Nevertheless, the company I work for is attempting to incorporate some AI stuff into the product, and while the ability to kick tasks off and get information back by asking questions in natural language looks absolutely great in demos, I'm concerned about how much oversight and checking will be required to make sure things are correct, vs if we just keep using humans.
I recognise what you and Dan are saying, and it certainly chimes with my experience. Whether it's bizarre shadow artefacts that Topaz AI creates, or ChatGPT making errors in a synopsis (to the extent of inventing plot lines in one case), it's not something I'd want to go all-in with. Which is why I worry about the government's direction. In my experience, the more you ask of the various products, the more likely they are to veer off course. The government sounds like it's intending to ask a huge amount. I anticipate some disasters, a vast waste of money and an awful lot of unhappy people. Hint: those unhappy people won't be the vendors of AI products and services.
We have Github CoPilot at work. In my experience it works really well if you use IDE integration and have it suggest complete blocks of code. It is still very dependent on context of course, which as far as I can tell in the case of JetBrains IDE integration is based on files you currently have open, rather than the loaded project. It's certainly a step up from the single variable/type auto-complete us software engineers are more familiar with.
For generation of code based on asking for a solution, it is more hit and miss and does often need the input refined to be able to generate the correct solution, but it's still miles better than doom scrolling Google search results for links to Stack Overflow, which Google is really bad for these days IMO
Anybiody worried about the chances of AI becoming self aware can relax a little in the knowledge that the developers of OpenAI at least have a predictable lack of insight and self awareness. Reports today suggest they are accusing the Chinese developers of DeepSeek of plagiarism. Without a trace of (understanding of) irony, they claim there is evidence that DeepSeek used OpenAI as a source for training.
Fully justified of course. Those pesky foreigners should have written it like OpenAI where the developers obviously wrote, drew, painted, filmed, sang, spoke every one of the hundreds of thousands, even millions of pieces of work used to train it, without even a hint of plagiarism.
I keep meaning to try and embrace ChatGPT (and other language models) more to try and figure out and reduce some work or tasks I'm doing.
At work we are very lacking in systems and processes. It's a nightmare. Someone new is trying to introduce a new system/solution which supposedly has some amount of AI.
I'm part of the small initial team on the project and currently the tasks they are looking to wrap in some form of "AI" are pretty simple interactions between systems that absolutely don't require AI. Just calculations based on predefined parameters to save time manually doing them, then feed the result to another system. (an MRP system with automated order sending).
Frustrates me when this phase of the project is claiming some sort of AI use case when it's not really true. Further phases could, but I'd still be very cautious of letting it loose. Probably mainly because a lot of the data being fed in from other parts of the business is generally shite - shit in, shit out.
Your situation sounds familiar. So few businesses of any kind pay any serious attention to the early stages in customer interaction (the online or other computer kind). Just thinking a bit about what somebody needs to to enter in a simple initial screen, which may be little more than a menu, and how that screen responds, is like some kind of magic to many people. And yet those tools have been there for years. I used to teach ISPF which is probably the most common way that developers interact with IBM mainframes, and although IBM provided a hugely rich variety of ways of prompting, validating and responding to every field on a menu, it was incredibly hard to get people to make use of them. A few careful decisions and it was easy to build a menu that had much of the functionality that today we would call AI. And probably more secure and fault tolerant.