Connect with us

AI

AI Weekly: Meet the people trying to replicate and open-source OpenAI’s GPT-3

Avatar

Published

on

In June, OpenAI published a paper detailing GPT-3, a machine learning model that achieves strong results on a number of natural language benchmarks. At 175 billion parameters — the part of the model that has learned from historical training data — it’s one of the largest of its kind. It’s also among the most sophisticated, with the ability to make primitive analogies, write in the style of Chaucer, and even complete basic code.

In contrast to GPT-3’s predecessors, GPT-2 and GPT-1, OpenAI chose not to open-source the model or training dataset, opting instead to make the former available through a commercial API. The company further curtailed access by choosing to exclusively license GPT-3 to Microsoft, which OpenAI has a business relationship with. Microsoft has invested $1 billion in OpenAI and built an Azure-hosted supercomputer designed to further OpenAI’s research.

Several efforts to recreate GPT-3 in open source have emerged, but perhaps the furthest along is GPT-Neo, a project spearheaded by EleutherAI. A grassroots collection of researchers working to open-source machine learning research, EleutherAI and its founding members — Connor Leahy, Leo Gao, and Sid Black — aim to deliver the code and weights needed to run a model similar, though not identical, to GPT-3 as soon as August. (Weights are parameters within a neural network that transform input data.)

EleutherAI

According to Leahy, EleutherAI began as “something of a joke” on TPU Podcast, a machine learning Discord server, where he playfully suggested someone should try to replicate GPT-3. Leahy, Gao, and Black took this to its logical extreme and founded the EleutherAI Discord server, which became the base of the organization’s operations.

“I consider GPT-3 and other similar results to be strong evidence that it may indeed be possible to create [powerful models] with nothing more than our current techniques,” Leahy told VentureBeat in an interview. “It turns out to be in fact very, very hard, but not impossible with a group of smart people, as EleutherAI has shown, and of course with access to unreasonable amounts of computer hardware.”

As part of a personal project, Leahy previously attempted to replicate GPT-2, leveraging access to compute through Google’s Tensorflow Research Cloud (TFRC) program. The original codebase, which became GPT-Neo, was built to run on tensor processing units (TPUs), Google’s custom AI accelerator chips. But the EleutherAI team concluded that even the generous amount of TPUs provided through TFRC wouldn’t be sufficient to train the GPT-3-like version of GPT-Neo in under two years.

GPT-Neo

EleutherAI’s fortunes changed when the company was approached by CoreWeave, a U.S.-based cryptocurrency miner that provides cloud services for CGI rendering and machine learning workloads. Last month, CoreWeave offered the EleutherAI team access to its hardware in exchange for an open source GPT-3-like model its customers could use and serve.

Leahy insists that the work, which began around Christmas, won’t involve money or other compensation going in either direction. “CoreWeave gives us access to their hardware, we make an open source GPT-3 for everyone to use (and thank them very loudly), and that’s all,” he said.

Training datasets

EleutherAI concedes that because of OpenAI’s decision not to release some key details of GPT-3’s architecture, GPT-Neo will deviate from it in at least those ways. Other differences might arise from the training dataset EleutherAI plans to use, which was curated by a team of 10 people at EleutherAI, including Leahy, Gao, and Black.

Language models like GPT-3 often amplify biases encoded in data. A portion of the training data is not uncommonly sourced from communities with pervasive gender, race, and religious prejudices. OpenAI notes that this can lead to placing words like “naughty” or “sucked” near female pronouns and “Islam” near words like “terrorism.” Other studies, like one published in April by Intel, MIT, and the Canadian Institute for Advanced Research (CIFAR) researchers, have found high levels of stereotypical bias in some of the most popular models, including Google’s BERT and XLNetOpenAI’s GPT-2, and Facebook’s RoBERTa. Malicious actors could leverage this bias to foment discord by spreading misinformation, disinformation, and outright lies that “radicalize individuals into violent far-right extremist ideologies and behaviors,” according to the Middlebury Institute of International Studies.

For their part, the EleutherAI team says they’ve performed “extensive bias analysis” on the GPT-Neo training dataset and made “tough editorial decisions” to exclude some datasets they felt were “unacceptably negatively biased” toward certain groups or views. The Pile, as it’s called, is an 835GB corpus consisting of 22 smaller datasets combined to ensure broad generalization abilities.

“We continue to carefully study how our models act in various circumstances and how we can make them more safe,” Leahy said.

Leahy personally disagrees with the idea that releasing a model like GPT-3 would have a direct negative impact on polarization. An adversary seeking to generate extremist views would find it much cheaper and easier to hire a troll farm, he argues, as autocratic governments have already done. Furthermore, Leahy asserts that discussions of discrimination and bias point to a real issue but don’t offer a complete solution. Rather than censoring the input data of a model, he says the AI research community must work toward systems that can “learn all that can be learned about evil and then use that knowledge to fight evil and become good.”

GPT-Neo

“I think the commoditization of GPT-3 type models is part of an inevitable trend in the falling price of the production of convincing digital content that will not be meaningfully derailed whether we release a model or not,” Leahy continued. “The biggest influence we can have here is to allow more low-resource users, especially academics, to gain access to these technologies to hopefully better study them, and also perform our own brand of safety-focused research on it, instead of having everything locked inside industry labs. After all, this is still ongoing, cutting-edge research. Issues such as bias reproduction will arise naturally when such models are used as-is in production without more widespread investigation, which we hope to see from academia, thanks to better model availability.”

Google recently fired AI ethicist Timnit Gebru, reportedly in part over a research paper on large language models that discussed risks such as the impact of their carbon footprint on marginalized communities. Asked about the environmental impact of training GPT-Neo, Leahy characterized the argument as a “red herring,” saying he believes it’s a matter of whether the ends justify the means — that is, whether the output of the training is worth the energy put into it.

“The amount of energy that goes into training such a model is much less than, say, the energy that goes into serving any medium-sized website, or a single trans-Atlantic flight to present a paper about the carbon emissions of AI models at a conference, or, God forbid, Bitcoin mining,” Leahy said. “No one complains about the energy bill of CERN (The European Organization for Nuclear Research), and I don’t think they should, either.”

Future work

EleutherAI plans to use architectural tweaks the team has found to be useful to train GPT-Neo, which they expect will enable the model to achieve performance “similar” to GPT-3 at roughly the same size (around 350GB to 700GB of weights). In the future, they plan to distill the final model down “an order of magnitude or so smaller” for easier inference. And while they’re not planning to provide any kind of commercial API, they expect CoreWeave and others to set up services to make GPT-Neo accessible to users.

As for the next iteration of GPT and similarly large, complex models, like Google’s trillion-parameter Switch-C, Leahy thinks they’ll likely be more challenging to replicate. But there’s evidence that efficiency improvements might offset the mounting compute requirements. An OpenAI survey found that since 2012, the amount of compute needed to train an AI model to the same performance classifying images in a popular benchmark (ImageNet) has been decreasing by a factor of two every 16 months. But the extent to which compute contributes to performance compared with novel algorithmic approaches remains an open question.

“It seems inevitable that models will continue to increase in size as long as increases in performance follow,” Leahy said. “Sufficiently large models will, of course, be out of reach for smaller actors, but this seems to me to just be a fact of life. There seems to me to be no viable alternative. If bigger models equals better performance, whoever has the biggest computer will make the biggest model and therefore have the best performance, easy as that. I wish this wasn’t so, but there isn’t really anything that can be done about it.”

For AI coverage, send news tips to Khari Johnson and Kyle Wiggers and AI editor Seth Colaner — and be sure to subscribe to the AI Weekly newsletter and bookmark our AI channel, The Machine.

Thanks for reading,

Kyle Wiggers

AI Staff Writer

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform
  • networking features, and more

Become a member

Source: https://venturebeat.com/2021/01/15/ai-weekly-meet-the-people-trying-to-replicate-and-open-source-openais-gpt-3/

AI

6 Best Artificial Intelligence Apps Making Human Lives Easier

Avatar

Published

on

It is no secret that Artificial Intelligence (AI) has gained a lot of popularity and taken the world by storm. Past studies have revealed that the global market of AI is predicted to reach from US$29.82 billion in 2019 to US$252.01 billion in 2025, growing at a CAGR of 42.7 percent.

“Research by Gartner identified that in the next few years, every application or service will incorporate AI at some level. ” 

Due to its benefits, AI is being used in different business functions and fields, including marketing, operations, finance, etc. 

Besides this, AI allows computers to extract valuable insights from large volumes of data, thereby helping perform complex tasks with ease. Moreover, AI-based apps have not only facilitated organizations, but have made human lives a lot easier. For example, rule-based automation and machine learning have simplified many routine jobs.   

Apart from business applications, AI has now gained popularity in the real world scenario and is being increasingly used in web-based and mobile applications. And for this reason, AI-enabled apps are being increasingly embedded in mobile operating systems that are available in the market.

We have created a list of the 6 best Artificial Intelligence apps that are making human lives easier. 

#1 Google Assistant 

The first one on our list is none other than the famous Google Assistant, which is an AI-driven virtual app generally available on smart home devices and mobile phones. The good thing about this AI app is that it supports different languages. This gives users a chance to interact with the app by using their mother tongue. 

Apart from this, the Google Assistant can perform all the possible tasks that can make your life easy and hassle-free. Not sure which ones? Have a look at the list below:

  • It can access the required information from your calendar. 
  • It can look up for information online, such as restaurant bookings, weather updates, etc. 
  • It can play your favorite music for you. 
  • It facilitates in making appointments and sending messages. 
  • It can open different apps on your phone or other smart devices. 

#2 Ada – Healthcare in Your Hand

Ada is another AI-based app that has a clean and interactive user interface and it helps you in understanding what are you suffering from. This app works like any other doctor’s appointment as you are required to share your basic details first and then details regarding the symptoms.

Based on the information that you provide, it prepares a medical report regarding your current condition, which includes a summary, potential causes, etc. You can also share that report as a PDF with your doctor or any medical expert for further diagnosis. 

Besides this, Ada is an accessible health advisor which is powered by a smart Artificial Intelligence engine. The good thing about this app is that it considers all important patient information, such as past medical history, risk factors, and more. And after gathering this data, it leverages machine learning and AI to provide accurate feedback.

#3 Hookt

Another one on our list is Hookt which is a platform that facilitates in chatting, making friends with people who share common interests. So, if you are somebody who is fond of making new friends in your area or while traveling, you should try the Hookt app. 

All in all, it is crucial to note that sometimes apps ask you to share your personal information, thus increasing the likelihood of scams. A study carried out by Business Insider revealed that 1 out of every 10 adults falls prey to a scam each year in the US. 

Therefore, you should be well aware of how to detect a fraud or scam as this can save your money. For instance, previously, many Vodafone users reported that they had to pay for the airG chat service even when they didn’t subscribe for it. However, airG was fast enough to identify this airG spam and took corrective actions immediately. Since not everyone is as responsible and authentic as airG, you should be beware. 

#4 Socratic 

AI and Robotics have not only revolutionized the way businesses are run, instead they are helping improve educational standards at schools considerably.  

“Socratic is one of the most popular AI-based apps for iOS and Android that helps students with their homework.” 

This app provides educational and informative resources, such as explanatory videos, definitions, etc. To use this app, users are required to take a photo of their homework, and then the app provides them with solutions and principles that can help solve the problems. 

The best thing about this app is that it is not subject-specific, instead, it provides help regarding different subjects, including Chemistry, English, Economics, Math, and Science. 

#5 YVA

YVA is a cloud-based app that is widely used in many different organizations. This app leverages AI for evaluating the performance of employees. Apart from this, YVA can be connected to instant messengers and corporate mail, thereby making it a one-stop solution. 

Another good thing about this app is that it conducts regular employee surveys, and analyzes the received information. This helps send the required warnings to employees and managers. Besides this, YVA helps prevent conflicts in teams, as it recognizes the competencies, like strengths, weaknesses, and leadership qualities of each employee. 

#6 Juke Deck

Juke Deck leverages the full potential of AI-based technologies for creating music tracks of different genres. However, the users of this app are required to identify the initial parameters of the composition, including the genre, tempo, duration, mood, etc. Once these variables are finalized, the Create Track Buttons helps process the track. 

The good thing is that you can either listen to the music that you have composed in a browser or download it on your computer. Revisions to the processed track can be done easily as well. Apart from this, Juke Deck does not ask for royalties, therefore you can use the work created as you wish. For example, you can post your created tracks on social media or YouTube. 

The Final Words 

In a nutshell, Artificial Intelligence has completely revolutionized the way we live and has made our lives a lot more convenient, on both – a personal and professional level. Therefore, you should try the above-mentioned AI-powered apps and enjoy their benefits.

The post 6 Best Artificial Intelligence Apps Making Human Lives Easier appeared first on Aiiot Talk – Artificial Intelligence | Internet of Things | Technology.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: http://www.aiiottalk.com/artificial-intelligence-apps-making-human-lives-easier/

Continue Reading

Aerospace

We Need to Build a Cloud for the Next Decade: Satya Nadella

Avatar

Published

on

As the world enters the second wave of digital transformation, we need to foundationally transform how the cloud can drive the next level of economic growth, Microsoft Corp. Chief Executive Satya Nadella said.

Nadella was speaking at the Microsoft Ignite 2021, the annual flagship event virtually attended by more than 100,000 global participants, including IT decision-makers, developers, data professionals, security professionals and other technology enthusiasts.

“The true test of technology has always been whether organizations can improve their time to value, increase agility, and reduce costs…but as the world recovers, it will require much more from technology and the cloud, in particular, to help address our most pressing challenges,” Nadella said.

Five key attributes will drive the next-generation of innovation in the cloud, Nadella said. “These are ubiquitous and decentralized computing, sovereign data and ambient intelligence, empowered creators and communities, expanded economic opportunity for the global workforce, and trust by design,” he said.

As businesses have accelerated their digital journeys over the past year, the demand for technology has significantly picked up to enable various use-cases like telehealth, remote manufacturing, and new ways of working from home. Microsoft believes the cloud has been the foundation to enable all of these.

Microsoft announced ‘Microsoft Mesh’, a new mixed reality platform built on, which enables geographically distributed teams to interact holographically with each other.

“With mixed reality technology, the digital and physical worlds have come together,” Nadella said.

“This has been the dream for mixed reality, the idea from the very beginning,” said Microsoft Technical Fellow Alex Kipman. “You can actually feel like you are in the same place with someone sharing content or you can teleport from different mixed reality devices and be present with people even when you’re not physically together.”

Microsoft also announced a new semantic search capability in Azure Cognitive Search, an Artificial Intelligence (AI)-powered cloud search service for mobile and web app development. This capability enables developers to deliver results based on user intent as opposed to a keyword-based search, which is the industry norm. Semantic search leverages some of the most advanced natural language models to improve the relevance and ranking of search results.

The post We Need to Build a Cloud for the Next Decade: Satya Nadella appeared first on ELE Times.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: https://www.eletimes.com/we-need-to-build-a-cloud-for-the-next-decade-satya-nadella

Continue Reading

Artificial Intelligence

AI Offers Powerful Insights into Video Content Production

Avatar

Published

on

AI, language processing, and deep learning are three major technologies that affect streaming services. They are already used to create content and the picture looks beautiful and detailed. If we exploit Artificial Intelligence in live streaming, this would change everything. Modern technology allows us to create automated actions, trigger real-time graphics, analyze audio, monitor social media sentiment and auto-sharing.

AI Leads to New Opportunities for Video Content Development

Some developers saw AI technology as an opportunity and created useful tools that help in video production. Let’s talk about 4 software variants that could be used for video production.

NotMP3

To process multimedia you should have a useful tool that will allow getting them without payment. Nowadays the best content provider is Youtube. That is why we decided to mention NotMP3 software, which uses cutting-edge AI technology. This sort will allow you to fulfill this task for free.

This is free software which is downloadable from the site. This is a simple but very powerful tool that can help you to get audio or video file of any quality you like. You can also get files from Soundcloud and transfer them to various formats.

The main advantage of NotMP3 software is that it is completely free and does not require creating an account. All you have to do is to install and use it.  

Transcriptive AI

The first program that actually uses AI technology is Transcriptive AI. This is a universal tool that is exploited to transcribe footage. It will make that footage searchable. You can use this software to make caption and paper edits.

The creator allows getting a free trial copy but if you want to continue using it, you will have to buy the product. This soft will be useful if you want to find some pieces of audio that have been mentioned in the video. You will also have accurate transcripts spending as little time as possible. The most powerful component is searchability – you can look for clips, sequences, and markers. This machine learning tool works like Google and you just mention the keywords.

Pixop

This is not a tool but a cloud service that exploits intuitive interface and AI technologies for video enhancement. The main advantage of this service is that it is very easy to use. You will not need any plugins and other tools. No subscription and fees are required. This service has been created to encourage creators to update their digital archives as easily as it is possible.

Users value Pixop because it provides a place where you can make video enhancement as fast as possible. This is a flexible and scalable could service which has been created to tackle every requirement. And the most important feature is the fact that it is powered by machine learning – just let AI do the work! Just sign up and use the online tool.

Imagen

This is a media management tool that is used for keeping media safe, secure, and easy to find. This program will help you to organize and unlock the full potential of your video files. You just open a video file and start editing it. This software uses a simple but effective technology to process a picture. Choose a frame, pause it and crop out the needed fragment. Then this picture could be saved on your device and used to create media products. You can add annotations for any image.

This tool could be ideal for content makers that are willing to organize and see the true value of their content. This service lets you focus on the important features of the picture. It is very good for business in content-making industry.

AI is Invaluable for Creating New Video Content

So, what is the best AI solution for content creation? At first, you will need a tool that would be able to download the high-quality video as fast as possible. That is why you will definitely need NotMP3 downloader and converter.

After that try Transcriptive AI program because this is professional software. It has lots of useful features. But it is not free.

If you are looking for the fastest way, try Pixop cloud service – it has all the needed functions. You can use Imagen as a free offline variant which will help you to create the best content.

The post AI Offers Powerful Insights into Video Content Production appeared first on SmartData Collective.

Checkout PrimeXBT
Source: https://www.smartdatacollective.com/ai-offers-insights-into-video-content-production/

Continue Reading

Artificial Intelligence

AI Offers Powerful Insights into Video Content Production

Avatar

Published

on

AI, language processing, and deep learning are three major technologies that affect streaming services. They are already used to create content and the picture looks beautiful and detailed. If we exploit Artificial Intelligence in live streaming, this would change everything. Modern technology allows us to create automated actions, trigger real-time graphics, analyze audio, monitor social media sentiment and auto-sharing.

AI Leads to New Opportunities for Video Content Development

Some developers saw AI technology as an opportunity and created useful tools that help in video production. Let’s talk about 4 software variants that could be used for video production.

NotMP3

To process multimedia you should have a useful tool that will allow getting them without payment. Nowadays the best content provider is Youtube. That is why we decided to mention NotMP3 software, which uses cutting-edge AI technology. This sort will allow you to fulfill this task for free.

This is free software which is downloadable from the site. This is a simple but very powerful tool that can help you to get audio or video file of any quality you like. You can also get files from Soundcloud and transfer them to various formats.

The main advantage of NotMP3 software is that it is completely free and does not require creating an account. All you have to do is to install and use it.  

Transcriptive AI

The first program that actually uses AI technology is Transcriptive AI. This is a universal tool that is exploited to transcribe footage. It will make that footage searchable. You can use this software to make caption and paper edits.

The creator allows getting a free trial copy but if you want to continue using it, you will have to buy the product. This soft will be useful if you want to find some pieces of audio that have been mentioned in the video. You will also have accurate transcripts spending as little time as possible. The most powerful component is searchability – you can look for clips, sequences, and markers. This machine learning tool works like Google and you just mention the keywords.

Pixop

This is not a tool but a cloud service that exploits intuitive interface and AI technologies for video enhancement. The main advantage of this service is that it is very easy to use. You will not need any plugins and other tools. No subscription and fees are required. This service has been created to encourage creators to update their digital archives as easily as it is possible.

Users value Pixop because it provides a place where you can make video enhancement as fast as possible. This is a flexible and scalable could service which has been created to tackle every requirement. And the most important feature is the fact that it is powered by machine learning – just let AI do the work! Just sign up and use the online tool.

Imagen

This is a media management tool that is used for keeping media safe, secure, and easy to find. This program will help you to organize and unlock the full potential of your video files. You just open a video file and start editing it. This software uses a simple but effective technology to process a picture. Choose a frame, pause it and crop out the needed fragment. Then this picture could be saved on your device and used to create media products. You can add annotations for any image.

This tool could be ideal for content makers that are willing to organize and see the true value of their content. This service lets you focus on the important features of the picture. It is very good for business in content-making industry.

AI is Invaluable for Creating New Video Content

So, what is the best AI solution for content creation? At first, you will need a tool that would be able to download the high-quality video as fast as possible. That is why you will definitely need NotMP3 downloader and converter.

After that try Transcriptive AI program because this is professional software. It has lots of useful features. But it is not free.

If you are looking for the fastest way, try Pixop cloud service – it has all the needed functions. You can use Imagen as a free offline variant which will help you to create the best content.

The post AI Offers Powerful Insights into Video Content Production appeared first on SmartData Collective.

Checkout PrimeXBT
Source: https://www.smartdatacollective.com/ai-offers-insights-into-video-content-production/

Continue Reading
Bioengineer5 days ago

Unburdening China of cancer: Trend analysis to assist prevention measures

Nano Technology5 days ago

CEA-Leti & Dolphin Design Report FD-SOI Breakthrough that Boosts Operating Frequency by 450% and Reduces Power Consumption by 30%: Joint Paper Presented at ISSCC 2021 Shows How New Adaptive Back-Biasing Technique Overcomes Integration Limits in Chip Design Flows

Energy5 days ago

New Columbia Solar raises $75 million, looks to develop 50 projects in D.C.

Energy5 days ago

Washington State awards low-income solar project grants

Energy5 days ago

Watch: IKEA completes 1.35 MW solar carport, with seven more underway

Energy5 days ago

Cleantech Roundup: SolarEdge, Tigo Energy, MIT, and more

AR/VR16 hours ago

‘Farpoint’ Studio Impulse Gear Announces a New VR Game Coming This Year

SPACS5 days ago

SPAC Frenzy Emboldens Silicon Valley Startups to Forgo Venture Funding: Report

Nano Technology5 days ago

Fantastic voyage: Nanobodies could help CRISPR turn genes on and off: Stanford researchers attached biological nanobodies to the CRISPR DNA editing tool in an experimental combo designed to extend the usefulness of two promising molecular tools

Bioengineer5 days ago

New research finds exercise may help slow memory loss for people living with Alzheimer’s dementia

Esports2 days ago

PowerOfEvil on TSM’s Spring Split playoff preparation: ‘A lot of things are going to change in the next couple of days’

Aerospace2 days ago

Astra’s 100-year plan: Q&A with CEO Chris Kemp

Esports2 days ago

How to download Pokemon Unite APK, iOS, and Switch

Crowdfunding4 days ago

Fintech Unicorn Brex Explains how Digital Commerce Startups can Scale Operations by Selling Wholesale

Aerospace1 day ago

Partners produce rotor blade 3D-printed tool on Ingersoll 3D printer

Crowdfunding4 days ago

Verifi Reveals that Nearly $31 Billion Is Lost Yearly to Transaction Disputes, which May Be Reduced via “Proactive Management”

Payments1 day ago

4-parter on Coinbase “IPO” – Part 1 = 5 Reasons Why It Matters

Cyber Security5 days ago

Yeezy Fans Face Sneaker-Bot Armies for Boost ‘Sun’ Release  

Cyber Security2 days ago

How you can get someone’s Snapchat password?

Aviation1 day ago

Uganda Airlines To Fly The Rare Airbus A330-800 To London Heathrow

Trending