Connect with us

AI

Blockchain Startups Raised over $4 Billion in VC Funding in Q2 2021

Published

on

Most blockchain-based startups have seen funding from venture backers, despite the current cryptocurrency market downturn, recording over $4 billion in Q2 alone.

This massive venture capital backing is in keeping with the established trend of VC funding for blockchain firms as investors look to be part of the new wave of disruption associated with decentralized finance.

VC Backers Continue to Dole Out Funding for Blockchain Startups

According to CNBC on Thursday (July 22, 2021), venture capital investors seem not to worry about the volatile nature associated with the crypto market, especially with the current slump in market prices. Bitcoin, which reached an all-time high )ATH) of over $63,000 back in April, is trading within the $33,000 range, losing over 50% of its ATH. Ether price has also suffered a slump after getting to over $4,000 in May.

Meanwhile, data from CB Insights, an analytics firm, revealed that the total funds received by different blockchain companies are $4.38 billion. The figure signals a more than 50% increase from Q1 2021, and almost a ninefold growth compared to Q2 2020.

In May, major fintech company Circle received $440 million from VC backers, making it the largest venture capital funding in a blockchain company. Meanwhile, Circle is planning to go public through an alliance with a special purpose acquisition company (SPAC) Concord Acquisition Corp. The merger, if successful, will put Circle’s valuation at $4.5 billion.


ADVERTISEMENT

Ledger, a cryptocurrency hardware wallet, raised the second-biggest round in Q1 2021 with $380 million. According to an interview with CNBC in December 2020, the company’s CEO Pascal Gauthier noted that the cryptocurrency market was gradually maturing, with institutional investors showing interest in the emerging industry.

Speaking to CNBC, CB Insights senior analyst, Chris Bendtsen :

“At the current rate, blockchain funding will shatter the previous year-end record — more than tripling the total raised back in 2018. Blockchain’s record funding year is being driven by the rising consumer and institutional demand for cryptocurrencies. Despite short-term price volatility, VC firms are still bullish on crypto’s future as a mainstream asset class and blockchain’s potential to make financial markets more efficient, accessible, and secure.”

Institutional Investors Seek Exposure to Crypto Industry

The record inflow of funding for blockchain firms is coming from both traditional VC funds and blockchain-focused funds alike. Some asset managers are even creating blockchain venture arms for both early and late-stage funding of projects in the industry.

As previously reported by CryptoPotato in June, venture capital giant Andreessen Horowitz announced the launch of a $2.2 billion cryptocurrency fund. According to the company, the new fund would be distributed across various crypto and blockchain startups.

Blockchain Capital raised $300 million for its Fund V LP back in May, with PayPal, Visa. hedge funds, and others participating in the capital raise.

Meanwhile, the trend is continuing in Q3 2021 with massive funding deals. Recently, major cryptocurrency derivatives platform FTX secured a record $900 million in its Series B funding, causing the company’s valuation to grow to $18 billion.

SPECIAL OFFER (Sponsored)

Binance Futures 50 USDT FREE Voucher: Use this link to register & get 10% off fees and 50 USDT when trading 500 USDT (limited offer).

PrimeXBT Special Offer: Use this link to register & enter POTATO50 code to get 50% free bonus on any deposit up to 1 BTC.

You Might Also Like:

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://coingenius.news/blockchain-startups-raised-over-4-billion-in-vc-funding-in-q2-2021-10/?utm_source=rss&utm_medium=rss&utm_campaign=blockchain-startups-raised-over-4-billion-in-vc-funding-in-q2-2021-10

AI

SaaS Platform Plannuh, which Focuses on Financial Management, Business Optimization, Secures $4.1 Million

Published

on

Plannuh, a software-as-a-service or SaaS firm automating core marketing leadership processes such as planning, financial management, and optimization of business outcomes, reveals that it has finalized a $4.1 million equity round that was led by Ripple Ventures.

Existing investors Glasswing Ventures, an early-stage VC firm investing in the next-gen of intelligent enterprise and frontier tech startups, and Gradient Ventures, Google’s AI-focused venture fund, also made contribution’s to Plannuh’s investment round, which comes as Plannuh sees “significant growth amid expanded product offerings.”

Plannuh, which assists businesses with enhancing the value of their marketing by consolidating various marketing plans, budgets, and metrics into a Cloud-powered system for greater visibility and collaboration, has “increased its annual recurring revenue more than six times in the last 12 months and added more than 50 customers.”

As noted in update shared with CI, the firm has now added key capabilities to the Plannuh app, such as a sophisticated new metrics “tracking system and enhanced ROI measurement, new integrations with HubSpot and Salesforce, and a major AI-driven expense ingestion technology release to automate expense capture and reconcile with financial systems.”

Plannuh’s management noted that they’ll use the proceeds to “accelerate the development of automated planning and budgeting workflows, including incremental integrations and AI-based forecasting and recommendations.”

Peter Mahoney, founder and CEO at Plannuh, stated:

“We have seen a huge wave of interest in our platform from leading marketers who value our agile approach to planning and executing their marketing agenda as a strategic advantage. The investment from Ripple Ventures, and our long-time investors Glasswing and Gradient will allow us to further accelerate customer acquisition and continue to enhance our product platform, with particular focus on additional integrations and automation.”

Matt Cohen, Managing Partner, Ripple Ventures, remarked:

“It is well known that finance and marketing teams often do not see eye to eye. This largely stems from the fact that finance teams are numbers-driven, and historically marketing teams have struggled to demonstrate the value that advertising campaigns have on revenue. Another major issue for marketing teams is that most of their planning, budgeting, and expense tracking is done manually, either on paper or in Excel.”

Cohen added that Plannuh aims to address these inefficiencies and “help quantify the effectiveness of marketing spend, thus reducing the friction between marketing and finance departments, and helping marketing teams get better results.” He also noted that this is what “makes us so excited to lead this investment round in Plannuh alongside an amazing group of investors.”

Rudina Seseri, Founder and Managing Partner, Glasswing Ventures, remarked:

“Plannuh’s AI-driven marketing automation platform is a game-changing solution to marketing’s planning crisis, effectively putting a stop to marketing budget waste. From pre-seed to today, we have seen how Plannuh’s exceptional team led by Peter Mahoney and Scott Todaro, has been able to execute on their plan to deliver a single source of truth for budgeting, planning, and execution, and realize incredibly strong customer traction.”

Zach Bratun-Glennon, Partner at Gradient Ventures, noted that Plannuh offers a robust  automation platform that brings together real-time financial data with marketing and sales outcomes, “delivering unique data and insights daily.”

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.crowdfundinsider.com/2021/09/180683-saas-platform-plannuh-which-focuses-on-financial-management-business-optimization-secures-4-1-million/

Continue Reading

AI

GitHub Copilot and the Rise of AI Language Models in Programming Automation

Published

on

GitHub Copilot and the Rise of AI Language Models in Programming Automation

Read on to learn more about what makes Copilot different from previous autocomplete tools (including TabNine), and why this particular tool has been generating so much controversy.


github-copilot.png

Should I Use Github Copilot?

 
If you are a software engineer, or count any of them among your circle of acquaintances, then you’re probably already aware at some level of Copilot. Copilot is GitHub’s new deep learning code completion tool.

Autocomplete tools for programmers are nothing new, and Copilot is not even the first to make use of deep learning nor even the first to use a GPT transformer. After all, TabNine sprung out of a summer project by OpenAI alum Jacob Jackson and makes use of the GPT-2 general purpose transformer.

Microsoft (which owns GitHub) has packaged their own IntelliSense code completion tool with programming products since at least 1996, and autocomplete and text correction has been an active area of research since the 1950s.

Read on if you’d like to learn more about what makes Copilot different from previous autocomplete tools (including TabNine), and why this particular tool has been generating so much controversy.

Copyright Controversy

 

Screenshot demonstrating GitHub Copilot’s eagerness to recite the fast inverse square root function from Quake III.
Screenshot of Armin Ronacher demonstrating GitHub Copilot’s eagerness to recite the fast inverse square root function from Quake III.

 

Since its inception, Copilot has fueled a heated discussion about the product and its potential copyright implications. In large part this is due to the way the model was trained. GitHub Copilot is based on OpenAI’s Codex, a variant of GPT-3 fine-tuned on code. GPT-3 is OpenAI’s 175 billion-parameter (Codex is apparently based on the 12-billion parameter version of GPT-3) general-purpose transformer, and of course any giant transformer needs a giant training dataset to be effective. GitHub is just the place to find such a dataset, and Copilot’s training dataset included all public code hosted by GitHub.

This is a principal source of controversy surrounding the project, surpassing the discussion about automating away software engineering and the impact of tools like Copilot on the future of programming. More technical information about the model and its limitations can be found in the paper on Arxiv.


Portrait of Edmond de Belamy.” Collective Obvious used open source code to generate the piece, later selling for over $400,000 at auction, much to the chagrin of Robbie Barrat, whose code they used.

 

Some programmers are simply upset that their code contributed to what is likely to become a paid product without their explicit permission, with a few commenters on hacker news discussing leaving the platform.

In some ways the reaction to Copilot echoes of the GAN-generated “painting” that sold for nearly half a million dollars at auction. The art piece was created on top of open source contributions with a lineage of multiple authors, none of whom received any compensation that we know of as reward for the noteworthy success of the work at auction.

The code used to produce the artwork, and potentially the pre-trained model weights as well, was made publicly available under a BSD license by the model’s author, Robbie Barrat, whose own work was based on previous open source projects and who later modified the license to disallow commercial use of the pre-trained weights. It’s understandable for programmers to be frustrated when left out of profitable uses of their work, but there’s more to the copyright controversy surrounding Copilot than that.

”github Copilot has, by their own admission, been trained on mountains of gpl code, so i’m unclear on how it’s not a form of laundering open source code into commercial works.-Twitter user eevee.

GitHub Copilot is demonstrably capable of reproducing extended sections of copyleft code, which many in the open source community consider a violation of the terms of licenses like GPL.

Copilot was trained on all public code, including permissive open source licenses like the MIT License. However, it also copyleft licenses like the Affero General Public License (AGPL) that allows use and modification, but requires modified works to be made available under the same license. In some interpretations, code generated by GitHub Copilot can be considered derivative of the original training data, and perhaps more problematically Copilot can sometimes reproduce code from the training dataset verbatim. That makes Copilot a trickier case than, say, the Google book-scanning precedent often cited as a cornerstone of fair use for scraping copyrighted data.

The discussion on potential legal issues continues with little consensus from either side of the debate for now, and the subject may very likely become an issue for the courts to resolve.

Even if we assume that Copilot is totally in the clear legally, there may be other risks to using the product. If Copilot’s training is considered fair use and its output is not considered derivative or copyright/copyleft infringing work, it could still produce output that easily fits the criteria for plagiarism in the context of something like a PhD student writing code for their thesis research. For the time being it may be a good idea to use Copilot carefully, but there’s another reason that Copilot is a trending topic of discussion: Copilot can give surprisingly good solutions to common programming tasks, and appears to be both quantitatively and qualitatively more capable than previous autocomplete tools.

How Good is GitHub’s Copilot?

 

Youtube video showing how Github copilot crushes leetcode interview questions
GitHub Copilot CRUSHES Leetcode Interview Questions! Source

 

Even if you’re not a programmer, you’ve probably had some experience with autocomplete in the form of predictive text on a mobile phone. This will automatically suggest the next word as you begin to type it, and may suggest a slightly longer continuation such as to finish the current sentence.

For most programming autocomplete tools the amount and complexity of suggestions is roughly similar to what you’d find in a mobile phone keyboard, but not all code completion tools use modern (aka deep learning) machine learning.

The default autocomplete in vim, for example, will simply offer a list of suggested completions based on the words that a user has entered previously. More recently developed code completion tools like TabNine or Kite are a little more sophisticated and can suggest the completion of the rest of a line or two. The Kite website suggests this is enough to make a programmer nearly twice as efficient in terms of the number of keystrokes used, but Github Copilot takes this one step further, albeit with a very long stride.

Copilot has similar completion capabilities to the standard language GPT-3 it is based on, and working with the code completion tool looks and feels similar to the style of “prompt programming” that GPT-3 experimenters have adopted when working with the model. Copilot can interpret the contents of a docstring and write a function to match, or given a function and the start of an appropriately named test function it can generate unit tests. That saves a lot more than 50% of a programmer’s keystrokes.

Taken to its logical conclusion, when Copilot works perfectly it turns the job of a software engineer into something that looks a lot more like constant code review than writing code.

Several programmer-bloggers with early access to the technical preview version have put Copilot to the test by essentially challenging the model to solve interview-level programming problems. Copilot is pretty impressive in how well it can solve these types of challenges, but not good enough to warrant using its output without carefully reviewing it first.

Several online software engineers have put the “AI pair programmer” (as GitHub puts it) to the test. We’ll go through some of the points identified in the Arxiv paper as scenarios where Codex falls short and try to find examples in the experiments conducted by programmers involved in the technical preview.

The HumanEval Dataset

 
OpenAI Codex is a ~12 billion parameter GPT-3 fine-tuned on code, with Codex-S being the most advanced variant of Codex itself. To evaluate the performance of this model, OpenAI built what they call the HumanEval dataset: a collection of 164 hand-written programming challenges with corresponding unit tests, the sort you might find on a coding practice site like CodeSignal, Codeforces, or HackerRank.

In HumanEval, the problem specifications are included in function docstrings, and the problems are all written for the Python programming language.

While an undifferentiated GPT-3 without code-specific was unable to solve any of the problems in the HumanEval dataset (at least on the first try), the fine-tuned Codex and Codex-S were able to solve 28.8% and 37.7% of problems, respectively. By cherry-picking from the top-100 attempts at the problems, Codex-S was further able to solve 77.5% of problems.

One way to interpret this is that if a programmer was using Codex, they could expect to find a valid solution to a problem (at roughly the level of complexity encountered in technical interviews) by looking through the first 100 suggestions, or even blindly throwing attempted solutions at a valid set of unit tests until they pass. That’s not to say a better solution can’t be found, if a programmer is willing to modify suggestions from Codex, in one of the first few suggestions.

They also used a more complicated estimator for solving each problem by generating around 200 samples and calculating an unbiased estimator of the proportion of samples that pass unit tests, which they report as “[email protected]” where k is 100, the number of samples being estimated. This method has a lower variance than reporting [email protected] directly.

The Best Codex Model Still Under-Performs a Computer Science Student

 
The authors of Codex note that, being trained on over 150GB in hundreds of millions of lines of code from GitHub, the model has been trained on significantly more code than a human programmer can expect to read over the course of their careers. However, the best Codex model (Codex-S with 12 billion parameters) still under-performs the abilities of a novice computer science student or someone who spends a few afternoons practicing interview-style coding challenges.

In particular, Codex performance degrades rapidly when chaining together several operations in a problem specification.

In fact, the ability of Codex to solve several operations chained together drops by a factor of 2 or worse for each additional instruction in the problem specification. To quantify this effect, the authors at OpenAI built an evaluation set of string manipulations that could operate sequentially (change to lowercase, replace every other character with a certain character, etc.). For a single string manipulation, Codex passed nearly 25% of problems, dropping to just below 10% for 2 string manipulations chained together, 5% for 3, and so on.

The rapid drop-off in solving multi-step problems was seen by an early Copilot reviewer Giuliano Giacaglia on Medium. Giuliano reports giving Copilot a problem description of reversing the letters in each word in an input string, but instead Copilot suggested a function that reverses the order of words in a sentence, not letters in a sentence of words (“World Hello” instead of “olleH dlroW”). Copilot did, however, manage to write a test that failed for its own implementation.

Although not sticking to the multi-step string manipulation paradigm used by Giuliano and OpenAI to test Copilot, Kumar Shubham discovered an impressive result when Copilot successfully solved a multi-step problem description that involved calling system programs to take a screenshot, run optical character recognition on the image, and finally extract email addresses from the text. That does, however, raise the issue that Copilot may write code that relies on unavailable, out-of-date, or untrusted external dependencies. That’s a point raised by OpenAI alongside the model’s susceptibility to bias, ability to generate security vulnerabilities, and potential economic and energy costs in the section of their paper discussing risks.

Other reviews of Copilot by YouTubers DevOps Directive and Benjamin Carlson found impressive results when challenging Copilot with interview-style questions from leetcode.com, including some that seemed significantly more complex than chaining together a series of simple string manipulations. The difference in the complexity of code that Copilot can generate and the complexity of problem specifications that Copilot can understand is striking.

Perhaps the prevalence of code written in the style of interview practice questions in the training dataset leads to Copilot overfitting those types of problems, or perhaps it is just more difficult to chain together several steps of modular functionality than it is to churn out a big chunk of complex code that is very similar to something the model has seen before. Poorly described and poorly interpreted specifications are already a common source of complaints for engineers and their managers of the human variety, so perhaps it should not be so surprising to find an AI coding assistant fails to excel at parsing complicated problem specifications.

Copilot Alternatives

 
As of this writing, Copilot is still limited to programmers lucky enough to be enrolled in the technical preview, but fear not: myriad of other code completion assistants (whether using deep learning or not) are readily available to try out and it’s not a bad time to reflect on what increasing automation might mean for software engineering.

Earlier we mentioned TabNine, a code completion tool based in part on OpenAI’s GPT-2 transformer. Originally built by Jacob Jackson and now owned by codota, TabNine was able to solve 7.6% of the HumanEval benchmark in the [email protected] metric used by OpenAI authors. That’s fairly impressive considering that TabNine was designed to be a more hands-on code completion solution, unlike Codex which was explicitly inspired by the potential of GPT-3 models to produce code from problem descriptions. TabNine has been around since 2018 and has both free and paid versions.

Kite is another code completion tool in the same vein as TabNine, with free (desktop) and paid (server) versions that differ in the size of the model used by a factor of 25. According to Kite’s usage statistics, coders choose to use the suggested completions often enough to cut their keystrokes in half compared to manually typing out every line, and Kite’s cite their users’ self-reported productivity boost of 18%. Going by the animated demos on their website, Kite definitely suggests shorter completions than both TabNine and Copilot. This differs in degree from TabNine, which suggests only slightly longer completions for the most part, but it’s qualitatively different from Copilot: Copilot can suggest extended blocks of code and changes the experience from choosing the best completion to code reviewing several suggested approaches to the problem.

Is Copilot Here to Take Your Job or Just Your Code?

 
GitHub Copilot has some software engineers joking that the automation they’ve been building for years is finally coming home to roost, and soon we’ll all be out of a job. In reality this is very unlikely to be the case for many years as there is more to programming than just writing code.

Besides, it’s an oft-repeated trope that even interpreting exactly what a client or manager wants in a set of software specifications is more of an art than a science.

On the other hand, Copilot and other natural language code completion tools like it (and trust us, more are coming) are indeed likely to have a big impact on the way software engineers do their jobs. Engineers will probably spend more time reviewing code and checking tests, whether the code under scrutiny was written by an AI model or a fellow engineer. We’ll probably also see another layer of meta to the art of programming as “prompt programming” of machine learning programming assistants becomes commonplace.

As cyberpunk author William Gibson put it all those years ago: “the future is already here — it’s just not evenly distributed.”

Copilot has also ignited a debate on copyright, copyleft, and all manner of open source licenses and the philosophy of building good technology, which is a discussion that needs to take place sooner rather than later. Additionally, most contemporary interpretations of intellectual property require a human author for a work to be eligible for copyright. As more code is written in larger proportions by machine learning models instead of humans, will those works legally enter the public domain upon their creation?

Who knows? Perhaps the open source community will finally win in the end, as the great-great-great successor to Copilot becomes a staunch open source advocate and insists on working only on free and open source software.

 
Bio: Kevin Vu manages Exxact Corp blog and works with many of its talented authors who write about different aspects of Deep Learning.

Original. Reposted with permission.

Related:


PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.kdnuggets.com/2021/09/github-copilot-rise-ai-language-models-programming-automation.html

Continue Reading

AI

GitHub Copilot and the Rise of AI Language Models in Programming Automation

Published

on

GitHub Copilot and the Rise of AI Language Models in Programming Automation

Read on to learn more about what makes Copilot different from previous autocomplete tools (including TabNine), and why this particular tool has been generating so much controversy.


github-copilot.png

Should I Use Github Copilot?

 
If you are a software engineer, or count any of them among your circle of acquaintances, then you’re probably already aware at some level of Copilot. Copilot is GitHub’s new deep learning code completion tool.

Autocomplete tools for programmers are nothing new, and Copilot is not even the first to make use of deep learning nor even the first to use a GPT transformer. After all, TabNine sprung out of a summer project by OpenAI alum Jacob Jackson and makes use of the GPT-2 general purpose transformer.

Microsoft (which owns GitHub) has packaged their own IntelliSense code completion tool with programming products since at least 1996, and autocomplete and text correction has been an active area of research since the 1950s.

Read on if you’d like to learn more about what makes Copilot different from previous autocomplete tools (including TabNine), and why this particular tool has been generating so much controversy.

Copyright Controversy

 

Screenshot demonstrating GitHub Copilot’s eagerness to recite the fast inverse square root function from Quake III.
Screenshot of Armin Ronacher demonstrating GitHub Copilot’s eagerness to recite the fast inverse square root function from Quake III.

 

Since its inception, Copilot has fueled a heated discussion about the product and its potential copyright implications. In large part this is due to the way the model was trained. GitHub Copilot is based on OpenAI’s Codex, a variant of GPT-3 fine-tuned on code. GPT-3 is OpenAI’s 175 billion-parameter (Codex is apparently based on the 12-billion parameter version of GPT-3) general-purpose transformer, and of course any giant transformer needs a giant training dataset to be effective. GitHub is just the place to find such a dataset, and Copilot’s training dataset included all public code hosted by GitHub.

This is a principal source of controversy surrounding the project, surpassing the discussion about automating away software engineering and the impact of tools like Copilot on the future of programming. More technical information about the model and its limitations can be found in the paper on Arxiv.


Portrait of Edmond de Belamy.” Collective Obvious used open source code to generate the piece, later selling for over $400,000 at auction, much to the chagrin of Robbie Barrat, whose code they used.

 

Some programmers are simply upset that their code contributed to what is likely to become a paid product without their explicit permission, with a few commenters on hacker news discussing leaving the platform.

In some ways the reaction to Copilot echoes of the GAN-generated “painting” that sold for nearly half a million dollars at auction. The art piece was created on top of open source contributions with a lineage of multiple authors, none of whom received any compensation that we know of as reward for the noteworthy success of the work at auction.

The code used to produce the artwork, and potentially the pre-trained model weights as well, was made publicly available under a BSD license by the model’s author, Robbie Barrat, whose own work was based on previous open source projects and who later modified the license to disallow commercial use of the pre-trained weights. It’s understandable for programmers to be frustrated when left out of profitable uses of their work, but there’s more to the copyright controversy surrounding Copilot than that.

”github Copilot has, by their own admission, been trained on mountains of gpl code, so i’m unclear on how it’s not a form of laundering open source code into commercial works.-Twitter user eevee.

GitHub Copilot is demonstrably capable of reproducing extended sections of copyleft code, which many in the open source community consider a violation of the terms of licenses like GPL.

Copilot was trained on all public code, including permissive open source licenses like the MIT License. However, it also copyleft licenses like the Affero General Public License (AGPL) that allows use and modification, but requires modified works to be made available under the same license. In some interpretations, code generated by GitHub Copilot can be considered derivative of the original training data, and perhaps more problematically Copilot can sometimes reproduce code from the training dataset verbatim. That makes Copilot a trickier case than, say, the Google book-scanning precedent often cited as a cornerstone of fair use for scraping copyrighted data.

The discussion on potential legal issues continues with little consensus from either side of the debate for now, and the subject may very likely become an issue for the courts to resolve.

Even if we assume that Copilot is totally in the clear legally, there may be other risks to using the product. If Copilot’s training is considered fair use and its output is not considered derivative or copyright/copyleft infringing work, it could still produce output that easily fits the criteria for plagiarism in the context of something like a PhD student writing code for their thesis research. For the time being it may be a good idea to use Copilot carefully, but there’s another reason that Copilot is a trending topic of discussion: Copilot can give surprisingly good solutions to common programming tasks, and appears to be both quantitatively and qualitatively more capable than previous autocomplete tools.

How Good is GitHub’s Copilot?

 

Youtube video showing how Github copilot crushes leetcode interview questions
GitHub Copilot CRUSHES Leetcode Interview Questions! Source

 

Even if you’re not a programmer, you’ve probably had some experience with autocomplete in the form of predictive text on a mobile phone. This will automatically suggest the next word as you begin to type it, and may suggest a slightly longer continuation such as to finish the current sentence.

For most programming autocomplete tools the amount and complexity of suggestions is roughly similar to what you’d find in a mobile phone keyboard, but not all code completion tools use modern (aka deep learning) machine learning.

The default autocomplete in vim, for example, will simply offer a list of suggested completions based on the words that a user has entered previously. More recently developed code completion tools like TabNine or Kite are a little more sophisticated and can suggest the completion of the rest of a line or two. The Kite website suggests this is enough to make a programmer nearly twice as efficient in terms of the number of keystrokes used, but Github Copilot takes this one step further, albeit with a very long stride.

Copilot has similar completion capabilities to the standard language GPT-3 it is based on, and working with the code completion tool looks and feels similar to the style of “prompt programming” that GPT-3 experimenters have adopted when working with the model. Copilot can interpret the contents of a docstring and write a function to match, or given a function and the start of an appropriately named test function it can generate unit tests. That saves a lot more than 50% of a programmer’s keystrokes.

Taken to its logical conclusion, when Copilot works perfectly it turns the job of a software engineer into something that looks a lot more like constant code review than writing code.

Several programmer-bloggers with early access to the technical preview version have put Copilot to the test by essentially challenging the model to solve interview-level programming problems. Copilot is pretty impressive in how well it can solve these types of challenges, but not good enough to warrant using its output without carefully reviewing it first.

Several online software engineers have put the “AI pair programmer” (as GitHub puts it) to the test. We’ll go through some of the points identified in the Arxiv paper as scenarios where Codex falls short and try to find examples in the experiments conducted by programmers involved in the technical preview.

The HumanEval Dataset

 
OpenAI Codex is a ~12 billion parameter GPT-3 fine-tuned on code, with Codex-S being the most advanced variant of Codex itself. To evaluate the performance of this model, OpenAI built what they call the HumanEval dataset: a collection of 164 hand-written programming challenges with corresponding unit tests, the sort you might find on a coding practice site like CodeSignal, Codeforces, or HackerRank.

In HumanEval, the problem specifications are included in function docstrings, and the problems are all written for the Python programming language.

While an undifferentiated GPT-3 without code-specific was unable to solve any of the problems in the HumanEval dataset (at least on the first try), the fine-tuned Codex and Codex-S were able to solve 28.8% and 37.7% of problems, respectively. By cherry-picking from the top-100 attempts at the problems, Codex-S was further able to solve 77.5% of problems.

One way to interpret this is that if a programmer was using Codex, they could expect to find a valid solution to a problem (at roughly the level of complexity encountered in technical interviews) by looking through the first 100 suggestions, or even blindly throwing attempted solutions at a valid set of unit tests until they pass. That’s not to say a better solution can’t be found, if a programmer is willing to modify suggestions from Codex, in one of the first few suggestions.

They also used a more complicated estimator for solving each problem by generating around 200 samples and calculating an unbiased estimator of the proportion of samples that pass unit tests, which they report as “[email protected]” where k is 100, the number of samples being estimated. This method has a lower variance than reporting [email protected] directly.

The Best Codex Model Still Under-Performs a Computer Science Student

 
The authors of Codex note that, being trained on over 150GB in hundreds of millions of lines of code from GitHub, the model has been trained on significantly more code than a human programmer can expect to read over the course of their careers. However, the best Codex model (Codex-S with 12 billion parameters) still under-performs the abilities of a novice computer science student or someone who spends a few afternoons practicing interview-style coding challenges.

In particular, Codex performance degrades rapidly when chaining together several operations in a problem specification.

In fact, the ability of Codex to solve several operations chained together drops by a factor of 2 or worse for each additional instruction in the problem specification. To quantify this effect, the authors at OpenAI built an evaluation set of string manipulations that could operate sequentially (change to lowercase, replace every other character with a certain character, etc.). For a single string manipulation, Codex passed nearly 25% of problems, dropping to just below 10% for 2 string manipulations chained together, 5% for 3, and so on.

The rapid drop-off in solving multi-step problems was seen by an early Copilot reviewer Giuliano Giacaglia on Medium. Giuliano reports giving Copilot a problem description of reversing the letters in each word in an input string, but instead Copilot suggested a function that reverses the order of words in a sentence, not letters in a sentence of words (“World Hello” instead of “olleH dlroW”). Copilot did, however, manage to write a test that failed for its own implementation.

Although not sticking to the multi-step string manipulation paradigm used by Giuliano and OpenAI to test Copilot, Kumar Shubham discovered an impressive result when Copilot successfully solved a multi-step problem description that involved calling system programs to take a screenshot, run optical character recognition on the image, and finally extract email addresses from the text. That does, however, raise the issue that Copilot may write code that relies on unavailable, out-of-date, or untrusted external dependencies. That’s a point raised by OpenAI alongside the model’s susceptibility to bias, ability to generate security vulnerabilities, and potential economic and energy costs in the section of their paper discussing risks.

Other reviews of Copilot by YouTubers DevOps Directive and Benjamin Carlson found impressive results when challenging Copilot with interview-style questions from leetcode.com, including some that seemed significantly more complex than chaining together a series of simple string manipulations. The difference in the complexity of code that Copilot can generate and the complexity of problem specifications that Copilot can understand is striking.

Perhaps the prevalence of code written in the style of interview practice questions in the training dataset leads to Copilot overfitting those types of problems, or perhaps it is just more difficult to chain together several steps of modular functionality than it is to churn out a big chunk of complex code that is very similar to something the model has seen before. Poorly described and poorly interpreted specifications are already a common source of complaints for engineers and their managers of the human variety, so perhaps it should not be so surprising to find an AI coding assistant fails to excel at parsing complicated problem specifications.

Copilot Alternatives

 
As of this writing, Copilot is still limited to programmers lucky enough to be enrolled in the technical preview, but fear not: myriad of other code completion assistants (whether using deep learning or not) are readily available to try out and it’s not a bad time to reflect on what increasing automation might mean for software engineering.

Earlier we mentioned TabNine, a code completion tool based in part on OpenAI’s GPT-2 transformer. Originally built by Jacob Jackson and now owned by codota, TabNine was able to solve 7.6% of the HumanEval benchmark in the [email protected] metric used by OpenAI authors. That’s fairly impressive considering that TabNine was designed to be a more hands-on code completion solution, unlike Codex which was explicitly inspired by the potential of GPT-3 models to produce code from problem descriptions. TabNine has been around since 2018 and has both free and paid versions.

Kite is another code completion tool in the same vein as TabNine, with free (desktop) and paid (server) versions that differ in the size of the model used by a factor of 25. According to Kite’s usage statistics, coders choose to use the suggested completions often enough to cut their keystrokes in half compared to manually typing out every line, and Kite’s cite their users’ self-reported productivity boost of 18%. Going by the animated demos on their website, Kite definitely suggests shorter completions than both TabNine and Copilot. This differs in degree from TabNine, which suggests only slightly longer completions for the most part, but it’s qualitatively different from Copilot: Copilot can suggest extended blocks of code and changes the experience from choosing the best completion to code reviewing several suggested approaches to the problem.

Is Copilot Here to Take Your Job or Just Your Code?

 
GitHub Copilot has some software engineers joking that the automation they’ve been building for years is finally coming home to roost, and soon we’ll all be out of a job. In reality this is very unlikely to be the case for many years as there is more to programming than just writing code.

Besides, it’s an oft-repeated trope that even interpreting exactly what a client or manager wants in a set of software specifications is more of an art than a science.

On the other hand, Copilot and other natural language code completion tools like it (and trust us, more are coming) are indeed likely to have a big impact on the way software engineers do their jobs. Engineers will probably spend more time reviewing code and checking tests, whether the code under scrutiny was written by an AI model or a fellow engineer. We’ll probably also see another layer of meta to the art of programming as “prompt programming” of machine learning programming assistants becomes commonplace.

As cyberpunk author William Gibson put it all those years ago: “the future is already here — it’s just not evenly distributed.”

Copilot has also ignited a debate on copyright, copyleft, and all manner of open source licenses and the philosophy of building good technology, which is a discussion that needs to take place sooner rather than later. Additionally, most contemporary interpretations of intellectual property require a human author for a work to be eligible for copyright. As more code is written in larger proportions by machine learning models instead of humans, will those works legally enter the public domain upon their creation?

Who knows? Perhaps the open source community will finally win in the end, as the great-great-great successor to Copilot becomes a staunch open source advocate and insists on working only on free and open source software.

 
Bio: Kevin Vu manages Exxact Corp blog and works with many of its talented authors who write about different aspects of Deep Learning.

Original. Reposted with permission.

Related:


PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.kdnuggets.com/2021/09/github-copilot-rise-ai-language-models-programming-automation.html

Continue Reading

AI

Nine Tools I Wish I Mastered Before My PhD in Machine Learning

Published

on

Nine Tools I Wish I Mastered Before My PhD in Machine Learning

Whether you are building a start up or making scientific breakthroughs these tools will bring your ML pipeline to the next level.


By Aliaksei Mikhailiuk, AI Scientist



Image by Author.

 

Despite its monumental role in advancing technology, academia is often ignorant of industrial achievements. By the end of my PhD I realised that there is a myriad of great auxiliary tools, overlooked in academia, but widely adopted in industry.

From my personal experience I know that learning and integrating new tools can be boring, scary, could put back and demotivate, especially when the current set up is so familiar and works.

Dropping bad habits can be difficult. With every tool outlined below I had to accept that the way I did things was suboptimal. However, in the process I have also learnt that at times results not seen in the moment pay off ten fold at a later stage.

Below I talk about the tools that I have found very useful for researching and building machine learning applications, both as an academic and an AI engineer. I group the tools in four sections by their purpose: environment isolation, experiment tracking, collaboration and visualisation.

Isolating Environments

 
 
Machine learning is an extremly fast developing field and hence commonly used packages are updated very often. Despite developers efforts, newer versions are often not compatible with their predecessors. And that does cause a lot of pain!

Fortunately there are tools to solve this problem!

Docker

 
 



Image by Author.

 

How many times did those NVIDIA drivers caused you trouble? During my PhD I had a university managed machine that was regularly updated. Updated overnight and without any notice. Imagine my surprise when the morning after the update I find out that most of my work is now incompatible with the latest drivers.

Although not directly meant for that, docker saves you from these especially stressful before the deadline misfortunes.

Docker allows to wrap software in packages called containers. Containers are isolated units that have their own software, libraries and configuration files. In a simplified view a container is a separate, independent virtual operating system that has means to communicate with the outside world.

Docker has a plethora of ready made containers for you to use, without extensive knowledge of how to configure everything yourself it is very easy to get started with the basics.

For those wanting to have a quick start, check out this tutorial. Also Amazon AWS has done a great job explaining why and how to use docker for machine learning here.

Conda

 
 
Reusing someones code became a new norm today. Someone creates a useful repository on github, you clone the code, install and get your solution without the need to write anything yourself.

There is a slight inconvenience though. When multiple projects are used together you run into package managing problem, where different projects require different versions of packages.

I am glad I discovered Conda not so late in my PhD. Conda is a package and environment management system. It allows to create multiple environments and quickly installs, run and update packages and their dependencies. You can quickly switch between isolated environments and always be sure that your project interacts only with the packages you expect.

Conda provides their own tutorial on how to create your first environment.

Running, tracking and logging experiments

 
 
Two essential pillars, without which getting a PhD in an applied field is close to impossible are rigour and consistency. And if you have ever tried to work with machine learning models you probably know how easy it is to loose track of the tested parameters. Back in the day parameter tracking was done in lab notebooks, I am certain these are still very useful in other fields, but in Computer Science we now have tools much more powerful than that.

Weights and biases

 
 



Snapshot of the wandb panel for a set of simple metrics — train loss, learning rate and average validation loss. Notice that you can also track system parameters! Image by Author.

 

experiment_res_1.csv
experiment_res_1_v2.csv
experiment_res_learning_rate_pt_5_v1.csv
...


Do these names look familiar? If so, then your model tracking skills should be stepped up. This was me in the first year of my PhD. As an excuse, I should say that I had a spreadsheet where I would log the details of every experiment and all associated files. However, it is still very convoluted and every change in parameters logging would inevitably impact the post-processing scripts.

Weights and biases (W&B/wandb) is one of the gems that I found quite late, but now use in every project. It lets you track, compare, visualize and optimize machine learning experiments with just a few lines of code. It also lets you track your datasets. Despite a large number of options I found W&B easy to set up and use with a very friendly web interface.

For those interested check out their quick set up tutorial here.

MLflow

 
 



Image by Author.

 

Similar to W&B, MLFlow provides functionality for logging code, models and datasets on which your model has been trained. Although I have used it solely for the purpose of logging data, models and code, it provides functionality well beyond that. It allows to manage the whole ML lifecycle, including experimentation, reproducibility and deployment.

If you want to quickly integrate it into your models, check out this tutorial. Databricks have also shared a very nice explanation of MLflow.

Screen

 
 
Leaving the experiments running overnight and hoping that your machine won’t go to sleep was my go to option in the first half a year of my PhD. When the work moved to remote I used to worry about the ssh session breaking — the code was running for several hours and almost converged.

I learnt about screen function rather late, and so couldn’t save myself from half backed results in the mornings. But in this case it is indeed better late than never.

Screen lets you launch and use multiple shell sessions from a single ssh session. The process started with screen can be detached from session and then reattached at a later time. So your experiments can be run in the background, without the need to worry about session closing, or terminal crashing.

The functionality is summarised here.

Collaboration

 
 
Academia is notorious for not having proper mechanisms for effective team management. To an extent this is justified by very strict requirements for personal contribution. Nevertheless the pace at which machine learning is progressing needs joint effort. Below are two rather basic tools that would be handy for effective communication, especially in the new realm of remote work.

GitHub

 
 
Pretty basic, huh? After seeing all the horror of how people track their code in accademia I cannot stress how important it is to be well versed in version control. No more folders named code_v1, code_v2.

Github provides a very useful framework for code tracking, merging and reviewing. Whenever a team is building a deep image quality metric each member could have its own branch of the code, working in parallel. Different parts of the solution can then be merged together. Whenever someone introduces a bug, it is dead easy to revert to the working version. Overall I rank git as the most important of all the tools I have mentioned in this article.

Check out this step by step guide on how to quickly start up.

Lucidchart

 
 
Lucidchart was introduced to me recently, before that I was using draw.io — a very simple interface for creating diagrams. Lucidchart is thousand times more powerful and has a much more versatile functionality. Its major strength is the shared space for collaboration and ability to make notes next to diagrams. Imagine a giant online whiteboard with a huge set of templates.

For a quick start check this tutorial page by Lucidchart.

Visualisation

 
 
Numerous paper submissions, especially unsuccessful ones, have taught me that presentation is often as important as the results. If the reviewer, who usually does not have much time, does not understand the text, the work is straightaway rejected. Images made in haste make a poor impression. Someone once told me: “If you cannot make a chart, how can I trust your results?”. I disagree with this statement, however, I do agree that the impression does matter.

Inkscape

 
 
A picture is worth a thousand words (in fact, correction 84.1 word).

Inkscape is a FREE software for vector graphics. In fact I was taught how to use it in my web-development course in my undergrad. However, I learnt how to enjoy it in full only during my PhD — working on those pretty pictures for the papers.

Of all the functionality that Inkscape provides especially valuable was TexText extension. With this package you can integrate your latex formulas seamlesly into an image.

There is a myriad of tutorials, however for the basic functionality I would recommend the ones provided by Inkscape team here.

Streamlit

 
 
Did you ever need to create a simple website to showcase you results or a simple machine learning application? In just few lines of python code it’s possible with Streamlit.

I found it particularly useful for paper supplementary materials, however it can be even more useful for easy deployment and showcasing project demos to clients.

For a quick start up check out this tutorial.

Summary and beyond

 
 
Finishing my PhD while positioning myself in industry was not easy. But it taught me several important lessons I wish I had at an earlier stage of my PhD.

The most important lesson is that curiosity and readiness to learn and change can greatly impact the quality of your work.

Below is the summary of the tutorials I have mentioned in each section:

DockerTutorial
CondaTutorial
Weights and biasesTutorial
MLflowTutorial
GitHubTutorial
ScreenTutorial
InkscapeTutorial
StreamlitTutorial
LucidchartTutorial

If you liked this article share it with a friend! To read more on machine learning and image processing topics press subscribe!

Have I missed anything? Do not hesitate to leave a note, comment or message me directly!

 
Bio: Aliaksei Mikhailiuk has a proven track record of researching, developing, deploying and maintaining machine learning algorithms in Computer Vision, Preference Aggregation and Natural Language Processing.

Original. Reposted with permission.

Related:


PlatoAi. Web3 Reimagined. Data Intelligence Amplified.
Click here to access.

Source: https://www.kdnuggets.com/2021/09/nine-tools-mastered-before-phd-machine-learning.html

Continue Reading
Esports24 mins ago

YouTube Gaming adds Theatre Mode for mobile, Member Milestone chat

Esports39 mins ago

StreamElements raises $100 million in investment funding, plans to speed up development of tools for platforms

Esports48 mins ago

Fracture removed from VALORANT competitive queue due to bug

Aviation50 mins ago

International borders to open before Christmas

Aviation50 mins ago

International borders to open before Christmas

Cleantech1 hour ago

Interview With Solarflux CEO & CTO On Solving The Problem Of Lack Of Fuels In Rural Areas With Solar

Cleantech1 hour ago

Interview With Solarflux CEO & CTO On Solving The Problem Of Lack Of Fuels In Rural Areas With Solar

Cleantech2 hours ago

China Pledges To End Financing Of Coal Power Plants In Other Countries

Cleantech2 hours ago

China Pledges To End Financing Of Coal Power Plants In Other Countries

Cleantech2 hours ago

Entergy Makes $11 Billion A Year But Failed To Protect Louisiana’s Power Grid During Ida — And Why

Cleantech2 hours ago

Entergy Makes $11 Billion A Year But Failed To Protect Louisiana’s Power Grid During Ida — And Why

Crowdfunding2 hours ago

Akoin’s AKN Token Now on BitMart, Improves Access for African Users

Crowdfunding2 hours ago

Akoin’s AKN Token Now on BitMart, Improves Access for African Users

Crowdfunding2 hours ago

Shares in MoneyLion to Trade on NYSE on September 23rd as SPAC Deal Approved

Crowdfunding2 hours ago

Shares in MoneyLion to Trade on NYSE on September 23rd as SPAC Deal Approved

Covid192 hours ago

Florida Makes Quarantine Optional For Students Exposed To COVID-19

Big Data2 hours ago

Facebook’s technology head Mike Schroepfer to step down

Big Data2 hours ago

Facebook’s technology head Mike Schroepfer to step down

Big Data2 hours ago

Yellen stressed importance of withdrawing digital services taxes in call with UK

Big Data2 hours ago

Yellen stressed importance of withdrawing digital services taxes in call with UK

Big Data2 hours ago

Apple to pay bonuses of up to $1,000 to store employees – Bloomberg News

Big Data2 hours ago

Apple to pay bonuses of up to $1,000 to store employees – Bloomberg News

Big Data2 hours ago

Intel chief plans to attend White House meeting on chip shortage

Big Data2 hours ago

Intel chief plans to attend White House meeting on chip shortage

Energy2 hours ago

Biosurfactants Market by Application and Geography | Global Forecast to 2025 | 17,000+ Technavio Research Reports

Cannabis2 hours ago

Under the Radar Cannabis Companies to Watch | Cannabiz Media

Cannabis2 hours ago

Under the Radar Cannabis Companies to Watch | Cannabiz Media

Esports2 hours ago

All the Knives in CS:GO Operation Riptide

Covid192 hours ago

The FDA OKs Pfizer-BioNTech Booster For People 65+ And At High Risk For Severe COVID

Esports3 hours ago

Secret Rare cards for upcoming Pokémon TCG sets Fusion Arts, Fusion Strike revealed

Trending