Connect with us


GPT-3 – A Game Changer For Legal Tech?




Tech circles have been lit up over the last few weeks following the unveiling by OpenAI of GPT-3, a new type of pretrained language model capable of generating natural language text and computer code with the most minimal of inputs. Could it be a game changer for legal technology, especially in relation to NLP tools that analyse text, as well as doc generation systems?

Artificial Lawyer was initially quite sceptical about GPT-3, (or to give you its full name: Generative Pretrained Transformer 3, an unsupervised Transformer language model). How was being able to create auto-generated pages of chimeric text that appear to be written by a real person going to be useful, especially in the legal world? Wasn’t this just a gimmick? 

So, AL contacted legal AI experts in California to get their views, which seemed apt as OpenAI is an artificial intelligence research laboratory, founded in San Francisco in 2015 by Elon Musk and other tech giants. Its CEO is Sam Altman, former president of the legendary startup accelerator, Y Combinator. Microsoft has also provided $1 billion in funding to help their work. I.e. this is a serious project.

But, what did the legal sector experts think?

First, Dan Rubins, a legal tech founder based in California, who is well-known in the market for having a sensible and level-headed approach to the field, with both a solid understanding of the law and coding. This is what he said:

‘I think GPT-3 will be quite impactful, but it’s the trend that really matters.

In the abstract sense, GPT-3 has already shattered benchmarks, but it really crossed the ‘uncanny valley’ for me when I saw this demo: 

This was the first time I saw a machine perform a task similar to what I do, all day, every day.

Plenty of technical challenges also lie ahead. For example, GPT-3 apparently costs between $4m-$12m to train, so we are unlikely to see the pace of iteration that we’ve seen with prior models, especially in domain specific applications like medicine and law. GPT-3 is a couple of orders of magnitude larger than its prior – 175B parameters vs. 1.5B for GPT-2.

Microsoft released a 17B parameter model last month, so there’s a kind of ridiculousness to current NLP model sizes. Researchers will apply pruning and other techniques to shrink the models, as happened with the initial release of BERT and subsequent release of DistilBERT, it just takes some time (funding), creativity (funding), and effort (funding).

In legal tech, few will be using a custom GPT-3 model anytime soon.’

Next, AL asked the team at NLP-driven doc analysis company, Evisort, who are also based in California, what they thought about this new capability. Was it going to be useful to legal tech companies? This is what Amine Anoun, Chief Technology Officer, said:

‘GPT-3 is the biggest advance in AI language models since its predecessor, GPT-2, was released in 2018. Trained with two orders of magnitude more parameters, it’s posed to beat many current accuracy benchmarks in tasks like natural language generation, named entity recognition, and question answering.

Even though it’s not specifically trained on legal language, GPT-3 can still be useful to lawyers. With few-shot learning, additional legal contracts and metadata can be passed into the pre-trained GPT-3 model, enabling it to quickly learn to generate new contracts or clauses.

For example, Evisort’s proprietary AI extracts key metadata and clauses from contracts, and can flag risk based on a user’s risk playbook. But with GPT-3, we can go one step further and suggest edits and redlines with less training data to mitigate those risks, increasing speed and efficiency during contract negotiation.

Another application of GPT-3 is the ability to support smart searches. A user can ask questions like ‘Can I terminate the contract at any time?’ or ‘What state law governs the agreement?’.

These queries are already possible today with a higher accuracy than human review, but with the amount of variation and complexity of contract language, GPT-3 could push the envelope on precision and recall even more.

It’s unlikely a technology like GPT-3 will fully replace contract drafting anytime soon, but it can augment the process of contract generation, analysis and e-discovery.

We’re very excited about the possible applications of GPT-3 at Evisort. By combining our specialized AI algorithms with a language model of this scale, we can help lawyers manage their contracts even more efficiently.’

And Jerry Ting, CEO and co-founder of Evisort, added: ‘We will be exploring its capabilities and testing it, as we do with any major developments in AI, but it’s too early to tell whether it’s directly applicable yet.

If we do use it, it’ll probably be in conjunction with our proprietary solutions as a bolt on to supplement all the inventions we’ve already created.’

Meanwhile, Joshua Browder – also based on the West Coast – and the founder of DoNotPay, sent a quick reply to say:

‘It’s truly amazing. I’m using it right now and we are building it into several of our products.’

So…..DoNotPay is going to try and leverage GPT-3. You read it here first.

So, there you go. Given their views, it looks like we should take GPT-3 seriously, but not get carried away.

It’s going to be very interesting over the next couple of years to see how legal tech companies use this (and future iterations of it….roll on GPT-4), especially those tapping NLP and NLG (natural language generation) systems to build their products.

Plus, some additional insights from UK-based legal tech company, Genie AI, via their CEO, Rafie Faruq, who is an expert in machine learning.

‘The main innovations of GPT-3 are two things. Firstly, an incredibly large model size (175 bn parameters compared to Microsoft’s Turing-NLG with just 17 bn parameters), which is partly due to new datasets such as nearly a trillion data points in the Common Crawl dataset.

Secondly, its few-shot learning achieving sometimes state of the art performance on a variety of NLP tasks. The ability of GPT-3 to generate language without model fine-tuning in a task-agnostic way is a monumental breakthrough in NLP. However, GPT-2 wasn’t released for public use for a long time.

Furthermore, even if GPT-3 is released beyond a mere API, hosting 175 bn parameters in a legal tech product, and expecting fast inference (and user experience for lawyers) is highly unlikely.

So, in summary, this is unlikely to be usable by legal tech companies in a production grade application for some time.

We already use bi-directional transformers (at Genie AI), and will certainly upgrade to GPT-3 if it is possible. We’ve signed up for the beta (not many people have access to GPT-3 right now) and will be playing with it as soon as it comes out, but as per the quote, at this stage we’re quite doubtful it’s actually useable in a legal tech product, because of the sheer size of it.

Predictions are likely to be very slow, and fine-tuning or retraining it on legal data is likely to be very hard due to the computational resource needed. Language modelling in the last few years has basically been a battle of who has the most computational resource (as has much of machine learning).’

Have you had a test of the software yet? Let AL know what you think.



Resiliency And Security: Future-Proofing Our AI Future




Deploying AI in the enterprise means thinking forward for resiliency and security (GETTY IMAGES)

By Allison Proffitt, AI Trends

On the first day of the Second Annual AI World Government conference and expo held virtually October 28-30, a panel moderated by Robert Gourley, cofounder & CTO of OODA, raised the issue of AI resiliency. Future-proofing AI solutions requires keeping your eyes open to upcoming likely legal and regulatory roadblocks, said Antigone Peyton, General Counsel & Innovation Strategist at Cloudigy Law. She takes a “use as little as possible” approach to data, raising questions such as: How long do you really need to keep training data? Can you abstract training data to the population level, removing some risk while still keeping enough data to find dangerous biases?

Stephen Dennis, Director of Advanced Computing Technology Centers at the U.S. Department of Homeland Security, also recommended a forward-looking posture, but in terms of the AI workforce. In particular, Dennis challenged the audience to consider the maturity level of the users of new AI technology. Full automation is not likely a first AI step, he said. Instead, he recommends automating slowly, bringing the team along. Take them a technology that works in the context they are used to, he said. They shouldn’t need a lot of training. Mature your team with the technology. Remove the human from the loop slowly.

Of course, some things will never be fully automated. Brian Drake, U.S. Department of Defense, pointed out that some tasks are inherently human-to-human interactions—such as gathering human intelligence. But AI can help humans do even those tasks better, he said.

He also cautioned enterprises to consider their contingency plan as they automate certain tasks. For example, we rarely remember phone numbers anymore. We’ve outsourced that data to our phones while accepting a certain level of risk. If you deploy a tool that replaces a human analytic activity, that’s fine, Drake said. But be prepared with a contingency plan, a solution for failure.   

Organizing for Resiliency

All of these changes will certainly require some organizational rethinking, the panel agreed. While government is organized in a top down fashion, Dennis said, the most AI-forward companies—Uber, Netflix—organize around the data. That makes more sense, he proposed, if we are carefully using the data.

Data models—like the new car trope—begin degrading the first day they are used. Perhaps the source data becomes outdated. Maybe an edge use case was not fully considered. The deployment of the model itself may prompt a completely unanticipated behavior. We must capture and institutionalize those assessments, Dennis said. He proposed an AI quality control team—different from the team building and deploying algorithms—to understand degradation and evaluate the health of models in an ongoing way. His group is working on this with sister organizations in cyber security, and he hopes the best practices they develop can be shared to the rest of the department and across the government.

Peyton called for education—and reeducation—across organizations. She called the AI systems we use today a “living and breathing animal”. This is not, she emphasized, an enterprise-level system that you buy once and drop into the organization. AI systems require maintenance, and someone must be assigned to that caretaking.

But at least at the Department of Defense, Drake pointed out, all employees are not expected to become data scientists. We’re a knowledge organization, he said, but even if reskilling and retraining are offered, a federal workforce does not have to universally accept those opportunities. However, surveys across DoD have revealed an “appetite to learn and change”, Drake said. The Department is hoping to feed that curiosity with a three-tiered training program offering executive-level overviews, practitioner-level training on the tools currently in place, and formal data science training. He encouraged a similar structure to AI and data science training across other organizations.

Bad AI Actors

Gourley turned the conversation to bad actors. The very first telegraph message between Washington DC and Baltimore in 1844 was an historic achievement. The second and third messages—Gourley said—were spam and fraud. Cybercrime is not new and it is absolutely guaranteed in AI. What is the way forward, Gourley asked the panel.

“Our adversaries have been quite clear about their ambitions in this space,” Drake said. “The Chinese have published a national artificial intelligence strategy; the Russians have done the same thing. They are resourcing those plans and executing them.”

In response, Drake argued for the vital importance of ethics frameworks and for the United States to embrace and use these technologies in an “ethically up front and moral way.” He predicted a formal codification around AI ethics standards in the next couple of years similar to international nuclear weapons agreements now.


Continue Reading


AI Projects Progressing Across Federal Government Agencies




The AI World Government Conference kicked off virtually on Oct. 28 and continues on Oct. 29 and 30. Tune in to learn about AI strategies and plans of federal agencies. (Credit: Getty Images)

By AI Trends Staff

Government agencies are gaining experience with AI on projects, with practitioners focusing on defining the project benefit and the data quality is good enough to ensure success. That was a takeaway from talks on the opening day of the Second Annual AI World Government conference and expo held virtually on October 28.

Wendy Martinez, PhD, director of the Mathematical Statistics Research Center, US Bureau of Labor Statistics

Wendy Martinez, PhD, director of the Mathematical Statistics Research Center, with the Office of Survey Methods Research in the US Bureau of Labor Statistics, described a project to use natural language understanding AI to parse text fields of databases, and automatically correlate them to job occupations in the federal system. One lesson learned was despite interest in sharing experience with other agencies, “You can’t build a model based on a certain dataset and use the model somewhere else,”  she stated. Instead, each project needs its own source of data and model tuned to it.

Renata Miskell, Chief Data Officer in the Office of the Inspector General for the US Department of Health and Human Services, fights fraud and abuse for an agency that oversees over $1 trillion in annual spending, including on Medicare and Medicaid. She emphasized the importance of ensuring that data is not biased and that models generate ethical recommendations. For example, to track fraud in its grant programs awarding over $700 billion annually, “It’s important to understand the data source and context,” she stated. The unit studied five years of data from “single audits” of individual grant recipients, which included a lot of unstructured text data. The goal was to pass relevant info to the audit team. “It took a lot of training, she stated. “Initially we had many false positives.” The team tuned for data quality and ethical use, steering away from blind assumptions. “If we took for granted that the grant recipients were high risk, we would be unfairly targeting certain populations,” Miskell stated.

Dave Cook, senior director of AI/ML Engineering Services, Figure Eight Federal

In the big picture, many government agencies are engaged in AI projects and a lot of collaboration is going on. Dave Cook is senior director of AI/ML Engineering Services for Figure Eight Federal, which works on AI projects for federal clients. He has years of experience working in private industry and government agencies, mostly now the Department of Defense and intelligence agencies. “In AI in the government right now, groups are talking to one another and trying to identify best practices around whether to pilot, prototype, or scale up,” he said. “The government has made some leaps over the past few years, and a lot of sorting out is still going on.”

Ritu Jyoti, Program VP, AI Research and Global AI Research lead for IDC consultants, program contributor to the event, has over 20 years of experience working with companies including EMC, IBM Global Services, and PwC Consulting. “AI has progressed rapidly,” she said. From a global survey IDC conducted in March, business drivers for AI adoption were found to be better customer experience, improved employee productivity, accelerated innovation and improved risk management. A fair number of AI projects failed. The main reasons were unrealistic expectations, the AI did not perform as expected, the project did not have access to the needed data, and the team lacked the necessary skills. “The results indicate a lack of strategy,” Joti stated.

David Bray, PhD, Inaugural Director of the nonprofit Atlantic Council GeoTech Center, and a contributor to the event program, posted questions on how data governance challenges the future of AI. He asked what questions practitioners and policymakers around AI should be asking, and how the public can participate more in deciding what can be done with data. “You choose not to be a data nerd at your own peril,” he said.

Anthony Scriffignano, PhD, senior VP & Chief Data Scientist with Dun & Bradstreet, said in the pandemic era with many segments of the economy shut down, companies are thinking through and practicing different ways of doing things. “We sit at the point of inflection. We have enough data and computer power to use the AI techniques invented generations ago in some cases,” he said. This opportunity poses challenges related to what to try and what not to try, and “sometimes our actions in one area cause a disruption in another area.”

AI World Government continues tomorrow and Friday.

(Ed. Note: Dr. Eric Schmidt, former CEO of Google is now chair of the National Security Commission on AI, today was involved in a discussion, Transatlantic Cooperation Around the Future of AI, with Ambassador Mircea Geoana, Deputy Secretary General, North Atlantic Treaty Organization, and Secretary Robert O. Work, vice chair of the National Security Commission. Convened by the Atlantic Council, the event can be viewed here.)


Continue Reading


5 Work From Home Office Essentials




Working remotely from home had been increasing in popularity, but it’s now become a necessity for many professionals due to the pandemic.

“Some companies are eager to reopen their doors and return to the office, but a large number of employer and employees are making the transitional work environment a permanent change.” 

They can’t guarantee their health and safety in a socially-crowded space, plus, companies are able to save tons of money they would have spent on their commercial lease or mortgage payments.

That’s not the say that working from home doesn’t come at its own costs, however. It can lead to a huge hit in productivity without the right equipment in place. To maximize your performance and efficiency in a remote setting, be sure to purchase these five office essentials.

1. Powerful PC

This one probably feels like an obvious pointer, but let’s knock it off our list. You won’t be able to get by with a make-shift work station and in today’s digital domain, your computer will be at the core of everything you do.

Never-ending loading wheels, delayed downloads, and slow rendering will add seconds to every task you do, so if your company didn’t provide you with a workhorse computer tower, you might look into investing in one yourself, then deduct the cost in your tax return.

Depending on your line of work, it might make more sense to go for a laptop vs a desktop computer. Unless your tasks demand super sophisticated software and large storage space, you can probably get by with a portable PC. That way, when coffee shops begin to reopen and allow patrons to sit inside, you can work on-the-go without feeling tethered to your desk.

2. Ergonomic Office Chair

If you’re looking at a long-term remote situation, it’s worth spending the big bucks on an ergonomic office chair. You should feel comfortably locked into your seat for eight hours a day—at least if you want to concentrate on your workflow, rather than the cramp in your back.

Shop around for an office chair that’s sophisticated in design and specifically built to hold the human body. Some stand-out features you should look out for include:

  • Targeted support around the lumbar spine
  • Adjustable height so you can adjust the seat as necessary for your arms to rest naturally on the keyboard
  • Swivel base to effortlessly turn your body, preventing neck strain
  • Cushioned seat to comfort your tailbone
  • Ventilated fabric that promotes airflow so you don’t feel overheated when sitting in the chair for several hours

You might have to pay a couple of hundred dollars for the best-of-the-line features, but there is another item that might qualify as an eligible tax deduction—just be sure to keep all your receipts organized with a document scanner in case the IRS raises their eyebrows and issues an audit.

3. Wireless Keyboard

If you want to type faster and feel better while you’re at it, then a wireless keyboard is clutch. They enable you to bring the keys closer, decreasing the extension length of your arms and accompanying shoulder strain.

“It also helps reduce the strain on your eyes by moving the bright screen farther away from your direct line of sight.” 

And, last but not least, the keys are placed in an ergonomic position for a more natural finger splay, with ample cushioning wrist cushioning that helps prevent overuse injuries such as a carpal tunnel.

4. Noise-cancelling Headphones

To truly get in the zone, you should block out distractions with headphones the cancel noise in your environment—especially if your work station is set up in a common area. Other tips to stay focused include installing a website blocker and leaving your cellphone on the other side of the room.

5. House plant or flowers

People are scientifically proven to be more productive when working near fresh flowers or lush greenery. The good news is that you don’t need to have a green thumb or natural lighting to achieve this effect—even artificial foliage can brighten your mood and improve your performance.

Working from home sometimes can feel like you’re locked inside all day, so bringing the outside world inside your space can help ward off burnout.

Take these tips with you into 2021 and set yourself up for success in your new home office setting.


Continue Reading
Covid193 hours ago

How followers on Instagram can help to navigate your brand during a pandemic

Cyber Security11 hours ago

How Was 2020 Cyber Security Awareness Month?

AR/VR12 hours ago

Sci-fi Shooter Hive Slayer is Free, Asks Players for Louisiana Hurricane Relief Donations Instead

AR/VR12 hours ago

AMD Announces Radeon RX 6000-series GPUs with USB-C “for a modern VR experience”

AI14 hours ago

Resiliency And Security: Future-Proofing Our AI Future

AI14 hours ago

AI Projects Progressing Across Federal Government Agencies

Blockchain16 hours ago

Kucoin and Revain Announce Partnership

AR/VR17 hours ago

Crowdfunded AR Startup Tilt Five Secures $7.5M Series A Investment

AR/VR17 hours ago

The Importance of XR Influencers

AR/VR17 hours ago

Head Back Underground in 2021 With Cave Digger 2: Dig Harder

AR/VR19 hours ago

Five All-New Multiplayer Modes Revealed for Tetris Effect: Connected

Crowdfunding20 hours ago

The Perfect Investment

AR/VR21 hours ago

Snapchat’s new Halloween AR Lenses Offer Full Body Tracking

Cyber Security21 hours ago

How the PS5 Will Completely Change Gaming As We Know It?

Cyber Security21 hours ago

Compromised Credentials used by Hackers to Access the Content Management System

Cyber Security21 hours ago

Which are the safest payment methods for online betting?

Cyber Security21 hours ago

How to stay safe if you’re using an Android device for betting?

Quantum23 hours ago

Bell nonlocality with a single shot

Quantum1 day ago

Optimization of the surface code design for Majorana-based qubits

Quantum1 day ago

Classical Simulations of Quantum Field Theory in Curved Spacetime I: Fermionic Hawking-Hartle Vacua from a Staggered Lattice Scheme

Ecommerce1 day ago

How Digital Transformation Will Change the Retail Industry

AR/VR1 day ago

Where to Change Quest 2 Privacy Settings and See Your VR Data Collected by Facebook

Cyber Security1 day ago

Cyber Security Prognostication Conversation

AR/VR2 days ago

Win a Huge The Walking Dead Onslaught Merch Bundle Including the Game

AR/VR2 days ago

Hold Your Nerve With These Scary VR Horror Titles

Blockchain News2 days ago

Ethereum City Builder MCP3D Goes DeFi with $MEGA Token October 28

Crowdfunding2 days ago

I dare you to ignore this trend…

Blockchain News2 days ago

Why Bitcoin’s Price Is Rising Despite Selling Pressure from Crypto Whales

AI2 days ago

5 Work From Home Office Essentials

AR/VR2 days ago

Gorilla Glass Maker Corning & Pixelligent Partner to Develop Optics for Consumer AR Headsets

AR/VR2 days ago

Skye’s Beautiful VR Funeral

AR/VR2 days ago

AR For Remote Assistance: A True Game Changer

Blockchain News2 days ago

Smart Contract 101: MetaMask

AR/VR2 days ago

Yupitergrad Adding PlayStation VR & Oculus Quest Support Jan 2021

AR/VR2 days ago

Hack & Slash Rogue-lite ‘Until You Fall’ Leaves Early Access on Steam & Oculus PC

AR/VR2 days ago

‘Elite Dangerous: Horizons’ Now Free to All Owners of the Base Game

Blockchain News2 days ago

New Darknet Markets Launch Despite Exit Scams as Demand Rises for Illicit Goods

Blockchain News2 days ago

Bitcoin Millionaires at an All-Time High as Analysts Warn of a Pullback Before BTC Moves Higher

Fintech2 days ago

The Impact of BPM On the Banking And Finance Sector

AR/VR2 days ago

Samsung & Stanford University are Developing a 10,000 PPI OLED Display