Study proves ChatGPT cannot rival conveyancing solicitors
Bird & Co. conveyancing solicitors carried out a study using ChatGPT to see if it could rival the work of conveyancing solicitors.
UK conveyancing solicitors, Bird & Co, puts ChatGPT to the test to see if it could answer the most common first-time buyer queries accurately, in place of a property solicitor. Here’s what we can conclude about the chatbot and its place in the legal sector.
The study by conveyancing experts, Bird & Co, started with a keyword search using Ahrefs to discover the top questions related to first-time buyers. Each question was then plugged into the AI chatbot and analysed by the team to establish whether it could rival advice a solicitor provides.
Whilst conducting the research, ChatGPT provided what looked like detailed and intricate answers to a lot of the questions posed. It even brought in government information, which gave the appearance of factuality.
However, Bird & Co found the details within the answers were not always accurate, and the information was often very reductive, without providing much substance.
More specifically, we found that ChatGPT had the following faults in answering questions that first time buyers may have:
-
- Provided very basic answers that lacked in-depth explanation.
- Did not ask follow-up any questions.
- Provided factually incorrect responses.
- Got dates and locations wrong.
- Contradicted itself.
More specifically, ChatGPT had a lot of problems answering questions surrounding more complex areas of the property sector, including Stamp Duty, first time buyer schemes, ISAs and grants.
When it came to answering, “How much is stamp duty in the UK for a first time buyer?”, the answer was incorrect and out of date, lacking explanations surrounding locations, dates and temporary structures in place for Stamp Duty, currently in place until March 2025. In fact, it completely ignored that these economic measures from 2022 existed, referring instead to 2017 measures.
For questions surrounding the Help to Buy ISA, the answer was somewhat reductive. Firstly, it contradicted itself in two separate answers, confusing the date in which the ISA became no longer available to apply for. It also did not make it clear that those who set up an account of this nature before 2019 are still able to add money to it, and use it to buy a house, up until 2029/30.
A conveyancing solicitor, who will remain up to date with all the financial and legal changes surrounding the property industry, would be able to provide all the information you’d need surrounding this topic.
They would also be sure to ask follow-up questions to prompt their client, and get all the information they need from them; something the chatbot did not do.
As a first-time home buyer, it can be incredibly tricky to navigate the processes involved with buying a home for the first time. The concern for solicitors lies with the fact ChatGPT is a completely free tool to use, and many people may look for short cut legal advice without taking its faults and limitations into consideration first.
Partner and Head of Property, Daniel Chard, says:
“The most striking issue with ChatGPT appearing to give legal advice – asides from providing incorrect answers – is that it’s not able to tailor any advice given to the person asking.
“This could easily lead first-time buyers down a path where they waste time and money if they take advice from chatbots instead of solicitors.
“It is clear AI tools like this could be useful in providing basic understanding to legal queries.
“It could even be useful as a starting point for many first-time buyers who are curious about the processes involved in buying a home, but responses should be taken as surface level knowledge.
“As AI technology becomes more sophisticated, we think solicitors should embrace the benefits that ChatGPT provides and utilise it to reduce workload.
“It could be that AI tools are used to answer very basic queries or to draft skeleton documents such as for wills or contracts.
“This could be utilised by legal professionals to cut down time and workload where applicable.
“However, after carrying out this study, it’s clear ChatGPT has a long way to go before it replaces any form of legal expert in giving actionable advice involving conveyancing.”
If you are looking to buy a property in 2023 and require a solicitor to get you started, Bird & Co can help. If you are planning to purchase, sell or remortgage your home in 2023, then get in touch with Bird & Co’s conveyancing solicitors by emailing [email protected], or calling 01476 591711.
Kindly shared by Bird & Co. Solicitors
Main article photo courtesy of Pixabay
