Connect with us


Gain customer insights using Amazon Aurora machine learning

In recent years, AWS customers have been running machine learning (ML) on an increasing variety of datasets and data sources. Because a large percentage of organizational data is stored in relational databases such as Amazon Aurora, there’s a common need to make this relational data available for training ML models, and to use ML models […]



In recent years, AWS customers have been running machine learning (ML) on an increasing variety of datasets and data sources. Because a large percentage of organizational data is stored in relational databases such as Amazon Aurora, there’s a common need to make this relational data available for training ML models, and to use ML models to make predictions in database-based applications. This post shows how to easily extract your production data from Aurora, train an ML model in Amazon SageMaker, and integrate the model inferences back into your production database and applications. It extends a popular ML use case, predicting customer churn, and demonstrates how to achieve the real business goal of preventing customer churn. We’ll use a large phone company as our setting.

At our telco company, our CEO called us all into a meeting. “We have around 15% of customers leaving our service, or “churning”, every year! Losing customers and then acquiring new customers is expensive. It’s a major drag on our annual results!  Can we predict which customers are likely to churn, and give them incentives to stay? Can we use machine learning (ML) to help?”

Here’s our ensuing discussion, condensed for brevity.

“Well,” said our ML engineer. “I understand that all of our customer data is stored in an Amazon Aurora relational database. If the DBA can get me the data, I can build an ML model to predict which customers will churn. I can use an Amazon SageMaker XGBoost Built In Model for that – it’s an algorithm commonly used for regression, classification, and ranking problems. And with SageMaker automatic model tuning I should get a pretty good model.”

The DBA said, “Sure! I can give you a dump of part of our production database. AWS offers S3 download from Amazon Aurora, so that makes it easy.”

The CEO roared, “I don’t want to predict who is leaving! I want to prevent them!”

“If you can tell us what factors are most tied to churn, we could build some targeted incentive programs,” said Marketing.

“Hmmm,” said our ML engineer. “Once I have the model, I can tell you the most important factors, or features, as we call them, used by the model. We call that ‘explainability’.“

Customer Service chimed in, “We can also look at our customer service chats for clues about each customer’s sentiment. However, the number of chats is too big for us to read manually. If an ML-based system can tell – in the first sentence of the chat, say – that a customer is not happy or they’re likely to churn, then we could offer them one of those incentives on the spot!”

“I can use Amazon Comprehend to tell you the customer’s sentiment. That’s easy,” said the ML engineer.

The DBA chimed in, “Then with Amazon Aurora Machine Learning, we can just write a SQL query that returns the customer sentiment and churn prediction along with the basic customer information at the time you start the chat. That way you’ll have all the customer details at your fingertips. We can then use the programs that Marketing gives us to offer an incentive to the likeliest churners.”

“Great!” said the CEO. “Make it so!”

In the rest of this blog, we’ll walk you through the steps required to do so.

Solution overview

In most organizations, a database admin (DBA) or an application developers works with a database, and the ML engineer works with the ML model. However, one person can perform all the steps to implement the solution we describe here. Because the solution is implemented using the Amazon SageMaker built-in XGBoost algorithm, more in-depth data scientist skills are not needed in this case.

The following diagram illustrates the AWS services used to implement the solution.

You get access to ML inferences from your production database with the following steps:

  1. The data of interest is originally stored in an Aurora MySQL (A similar process can be used for the PostgreSQL-compatible edition of Aurora.) The DBA uses a SQL SELECT INTO OUTFILE S3 statement to unload data from Aurora to an S3 bucket.
  2. The ML engineer uses a Jupyter notebook hosted on an Amazon SageMaker notebook instance to read and explore the data from Amazon S3. AWS Secrets Manager is used to securely hold the Aurora access credentials.
  3. From the Jupyter notebook, the ML engineer partitions the data into separate sets of data for use in training and testing the ML model. The ML engineer writes the training data back to Amazon S3 to use during model training. (You could also perform partitioning during the initial data unload from Aurora.)
  4. From the same notebook, the ML engineer starts a training job using the Amazon SageMaker built-in XGBoost algorithm.
  5. Amazon SageMaker reads the training data from Amazon S3.
  6. Amazon SageMaker trains the model.
  7. The ML engineer checks the model characteristics. When satisfied, she creates an Amazon SageMaker endpoint to serve predictions from the trained model and sets up the connection between the database and the endpoint.
  8. The application (or the DBA) executes a SQL query in Aurora.
  9. Aurora passes on the request for the churn prediction to the Amazon SageMaker endpoint.
  10. Aurora passes on the request for the sentiment prediction on text to Amazon Comprehend.
  11. Aurora assembles the results and returns them with data from its own tables, as a SQL query result.

This post uses a dataset from the AWS Samples Customer Churn sample notebook. It’s publicly available and is mentioned in the book Discovering Knowledge in Data by Daniel T. Larose. It’s attributed by the author to the University of California Irvine Repository of Machine Learning Datasets.

What makes our life much easier is that Aurora has a feature called Aurora machine learning, which was launched at re:Invent 2019. This feature is a pre-built integration with Amazon SageMaker and Amazon Comprehend that calls these services within SQL queries and takes care of transferring the data efficiently. For instructions on configuring Aurora ML capabilities, see Enabling Aurora Machine Learning. This post creates and configures these components for you via an AWS CloudFormation template. You can also find a copy of all the code in AWS Samples: Gain Customer Insights.

Setting up the solution infrastructure

To set up the infrastructure components and configure the connections, launch a predefined CloudFormation stack. The stack sets up the following resources:

  • An Amazon VPC in which to run the components, and two private subnets and one public subnet with a NAT gateway.
  • An Amazon Aurora cluster containing a single database instance, and a security group that is configured to give access to the Amazon SageMaker notebook instance. The cluster resides in the private subnet. This database instance isn’t set up as a production configuration. For more information, see Category: Amazon Aurora and AWS Quick Starts for Aurora. The stack also creates a DB cluster parameter group with appropriate parameter settings, specifically the following:
    • aws_default_comprehend_role is set to an IAM role that gives Aurora permission to call the Amazon Comprehend DetectSentiment and BatchDetectSentiment APIs.
    • aws_default_sagemaker_role is set to an IAM role that gives Aurora permission to invoke the Amazon SageMaker endpoint that we’ll use for our model.
    • aws_default_s3_role is set to an IAM role that permits access to the S3 bucket we’ll use for our data.
  • An Amazon SageMaker notebook instance with the associated components:
    • An IAM role for use by the notebook instance. The IAM role has the managed role AmazonSageMakerFullAccess, which includes access to S3 buckets containing SageMaker in the name. This role may have more permissions than you wish to use in production.
    • A security group used for the notebook instance.
    • An Amazon SageMaker lifecycle configuration that copies in three predefined Jupyter notebooks with the sample code. It also creates a file called, which contains the values of key resources created by the stack, such as the S3 bucket name and the secret name that contains the Aurora hostname and the password. This file is used to pass these values to the Jupyter notebooks.
  • An Amazon SageMaker endpoint. The endpoint is listed in the IAM role specified in the DB cluster parameter group parameter aws_default_sagemaker_role.
  • An S3 bucket with SageMaker in the name so the Amazon SageMaker default IAM role can use it.
  • A Secrets Manager secret, with the database username, password, and hostname. Using the secret means that the username and password aren’t stored in the notebook.

To see this solution in operation in the US East (N. Virginia) Region, launch the stack from the following link. The total solution costs approximately $1.00 as a one-time charge plus $1.15 per hour to run. Remember to delete the AWS CloudFormation stack when you’ve finished with the solution to avoid additional charges.

To run the stack, complete the following steps:

  1. Choose Launch Stack and choose Next.
  2. Update the following parameters for your environment, or leave the defaults:
    1. DatabaseAdministratorUsername
    2. SageMakerInstanceType
    3. RDSInstanceType
  3. Choose Next.
  4. Select I acknowledge that AWS CloudFormation might create IAM resources with custom names.
  5. Choose Create.
  6. Wait for the CloudFormation stack to reach a status of CREATE_COMPLETE (up to 15–20 minutes).
  7. Check the Outputs tab for the resources created. See the following screenshot.
  8. On the Amazon SageMaker console, choose Notebook instances.
  9. Locate the notebook that starts with PreventChurnSMNotebook- in the list of notebook instances.
  10. Choose Open JupyterLab.

You should see a list of files copied into the notebook for you, similar to the following screenshot.

The following sections walk you through each notebook.

Setting up and unloading the data

Setting up the Amazon Aurora database and saving the data to Amazon S3 for use in ML are steps that the DBA would normally perform, often using a SQL client. For consistency, this post demonstrates the steps in a Jupyter notebook using mysql.connector Python module.

The first notebook, part_1_preventing_customer_churn_Amazon_Aurora_setup.ipynb, is divided into two sections. For a pre-executed HTML version, see Preventing Customer Churn, Part 1. Connect to Amazon Aurora MySQL database, data loading and extraction.

In this section, we fetch some sample customer data. Each record is labeled if this customer churned or not. We write this data out to Amazon S3. We create a table in a database to hold the data, then use a LOAD DATA FROM S3 statement to bulk load the data into an Aurora database table. We also store some historical customer messages into a second table.

Now we’re ready to walk through the use case.

In Section 2, we begin:  The DBA has just received the request: “Please export the customer data to S3, so the data scientist can explore the reason for data churn. Thanks!”

Luckily, there’s an Amazon Aurora feature that makes it easy: Saving Data from an Amazon Aurora MySQL DB Cluster into Text Files in an Amazon S3 Bucket. A single SQL statement, and the DBA has done her part! Customer data, including summaries of their telephone plans, usage, and customer calls, are all now in S3. We’re ready to move on to the second notebook.

Running the ML algorithm and reviewing model results

The second notebook walks you through training an Amazon SageMaker XGBoost model on the customer data and using hyperparameter optimization (HPO) to find the best model. It also explores the features used in the best-performing model to understand the most impactful features. This is the information Marketing needs in order to put together an incentive program.

For the data exploration and training code, see the notebook part_2_preventing_customer_churn_XGBoost.ipynb. (If you see the message select a kernel when starting this notebook, choose conda_python3.) For a pre-executed HTML version, see Preventing Customer Churn, Part 2. Building the ML Model.

In this notebook, we build two XGBoost models. The first model uses all the features available in the customer data. After dropping several highly-correlated columns and converting categorical variables (such as state) to one-hot vectors, the training data has around 70 columns.

This post presents a sample run. In this run we saw a training AUC of 0.949, and a validation AUC of 0.884 for the first model. In all cases, due to randomization inherent in XGBoost, the details vary slightly from run to run. This is most likely when several features have similar predictive values and can be substituted for each other with similar results, which leads to similar model performance.

We then load the XGBoost trained model into our notebook to analyze the features that it used to achieve these results. First, we’ll use the built-in feature importance plots from XGBoost. The following graph shows gain, which is the improvement in accuracy brought by a feature to the branches it’s on.

The following graph shows cover, which measures the relative quantity of observations concerned by a feature.

A few features, such as cust_service_calls, day_mins, and int_plan_no (a binary feature; that the customer has no international plan), are amongst the key drivers of customer churn. Feature importance tells you that this feature is influential but doesn’t tell you the direction (does it increase or decrease churn). The direction can often be inferred through business sense, or by evaluating the trees generated by the model, as shown below.

To address that question, this post presents a few of the trees generated. It’s not generally feasible to explore all trees, but it can be helpful to verify that the data the model uses and the model itself are both reasonable. This can be invaluable in identifying data extraction errors, spurious patterns, or the need for feature engineering.

The following diagram shows the first tree that XGBoost generated.

Interesting! The first split is on customer_service_calls of 3.5. In this image, we see the potential to use these trees to suggest or understand different kinds of churners. We can easily plot additional trees, or pull a list of the key splits, such as the top few splits of the various trees. This information may provide Marketing with some ideas for different churn profiles amongst customer segments, which can lead to different targeted incentive programs.

A little data exploration supports the idea that four or more calls shift the percentage of churners (see the following screenshot). We’ll likely want to intervene before they make that fourth phone call.

In some cases of using the XGBoost algorithm—primarily where the data has little information, or where the model isn’t converging well—some or even many or most of the trees may contain only a single leaf, and so don’t add any decision power to the model. In others, we’ve seen that we need to expand the hyperparameter bounds we’re using due to the inherent complexity in the data. To see if that’s affecting this particular model, we calculate the depth of each tree, as in the following bar plot.

In this test run, several trees contain only a leaf, so it appears that further optimization is possible.

We also want to get a sense of which features influence the model by seeing how the features are used across the trees. The following graph shows the model features and the number of each that has been used to split a tree during training.

In the preceding graph, the majority of features aren’t even used in the tree splits. The following table shows the features that are used at least one time in the run and the number of times they’re used.

Feature Number of Times Used
day_mins 8
cust_service_calls 7
eve_mins 7
int_plan_no 6
int_charge 6
vmail_msg 6
night_mins 5
int_calls 4
day_calls 1

Here, we see an opportunity! Whereas the training data had around 70 features, this model only uses 9 features. Due to the randomized nature of XGBoost, the exact number of times used and the rarely used features (small number of splits) change from run to run. A later section explores simplifying the model and thereby simplifying the calls from the production database, passing much less data to the model.

Evaluating the models

Next, we’ll evaluate our model against our test data. To test the model against the held-back test data, we deploy our model to an Amazon SageMaker endpoint created during the launch of the CloudFormation stack. This endpoint represents the inference functionality of Amazon SageMaker. For simplicity, this post uses a predefined endpoint_name. The CloudFormation template setup specified this endpoint_name in an IAM role and set the Aurora DB cluster group parameter aws_default_sagemaker_role to this IAM role. This combination of settings gives Aurora permission to call the Amazon SageMaker endpoint that we’ll be creating here. We can then look at statistics such as the number of correct predictions and the accuracy of the model.

Now, we build an updated model using the same approach, but with only the features used in our first model. We evaluate it against the full model using standard ML measures such as accuracy, recall, and precision, as in the following table.

We can also assess the ROC curve for both models, as in the following graph.

The updated model provides results close to the original model; on some runs it performs slightly better, on others, slightly worse.

Because the difference between the two models is below our margin of error, and the updated model requires far less data to be passed in the call (9 rather than 70 features), we will use the updated model.

Assessing business impact

We can also assess the model performance by looking at the prediction scores and refining the threshold used to decide if someone is a churner. It’s common to treat this as a binary classification, but the real world is less binary. People are considered likely to churn for some time before they actually churn. Loss of brand loyalty occurs some time before someone buys from a competitor.

The following graph shows that the continuous valued predictions coming from the model tend to skew toward 0 or 1, but there is sufficient mass between 0.1 and 0.9 that adjusting the cutoff should indeed shift many customers’ predictions. For now, we’ll take a simple approach of optimizing the threshold assuming a single incentive, to get a starting point for future analysis.

At this point, Marketing has not yet told us the incentives they’d like to use, so let’s assume a retention incentive of $50, for customers that our model identifies, and a cost of $500 for each false negative. We can use these numbers in a cost optimization model to identify the optimal threshold.

The following graph shows how setting a churn threshold too low (below 0.1) results in costs skyrocketing as all customers are given a retention incentive. Additionally, setting the threshold too high (0.7 or above) results in too many lost customers, which ultimately grows to be nearly as costly. In between, there is a large gap where perhaps some more nuanced incentives would create better outcomes.

Using these incentive numbers, we can minimize the overall cost at $5950 by setting the cutoff to 0.12, which is substantially better than the $20,000 or more the business can expect to lose by not taking any action. These numbers can be used by Marketing as a starting point for their incentive planning exercise.

Developing incentives

The ML engineer now returns to Marketing with the results of her analysis.

The feature importance charts showed that a small number of features make up most of the churn prediction: the number of customer service calls, use of an international plan, and number of minutes of calls during the day.

The feature importance charts show which features in the XGBoost model contribute most to the model performance, but they don’t specify which direction or at what values these variables cause a change in outcome. Inspecting the trees can give us some insight. For the tree shown earlier and the associated churn probabilities, making a fourth customer service call seems to signal a major shift in the odds of someone becoming a churner. Another segmentation seems to be at around 3 hours of calls a day.

Based on these insights, Marketing can develop a set of incentives to offer customers. They can choose several methods of rolling out incentives, such as targeting specific customers via mailings or other outreach methods. Here, we focus on intervention at the time of the customer’s next call to customer service. Key to using these incentives is the ability for Customer Service to assess each customer call and identify and make a relevant offer if appropriate. The next section shows how to implement this capability.

For this use case, we begin with some simple heuristics, based on the customer’s sentiment during the call and their use of an international plan.

Using ML capabilities in production

With the ML model trained and deployed, and advice from Marketing on the incentive programs to use, we’re now ready to put the pieces together.

The third notebook, part_3_preventing_customer_churn_inferences_from_Amazon_Aurora.ipynb, shows how to connect to and get the results of the ML capabilities—Amazon SageMaker and Amazon Comprehend—as part of a SQL query to Aurora. With both of these pieces of information in hand, we can make an on-the-fly decision about what incentive program to offer the customer. For a pre-executed HTML version, see Preventing Customer Churn, Part 3. Inference from Amazon Aurora.

We can perform all this work on Aurora, and will likely incorporate these queries on a backend of an internal application so customer service can in real time understand the risk of a customer churning based on a message from that customer and that customer’s details.

First, we’ll call Amazon Comprehend from Aurora. That way, we can also get an assessment of the customer’s sentiment in their message to customer service. The CloudFormation template already gave Aurora permission to call Amazon Comprehend by adding an IAM role to the DB cluster parameter group parameter aws_default_comprehend_role. Now, calling Amazon Comprehend is as simple as issuing a SQL query, as shown below.

sql = """SELECT message, aws_comprehend_detect_sentiment(message, 'en') AS sentiment, aws_comprehend_detect_sentiment_confidence(message, 'en') AS confidence FROM {} LIMIT 1;""".format(customer_msgs_table) [('Thank you very much for resolving the issues with my bill!', bytearray(b'POSITIVE'), 0.9981661438941956)]

The query response lists the original message and two outputs from Amazon Comprehend: the sentiment (POSITIVE), and the confidence level (in the preceding code, very high).

Now we’re ready to connect our Amazon SageMaker model. To do so, we create a function to call the Amazon SageMaker endpoint. For instructions, see Enabling Aurora Machine Learning. In Aurora, we create the SQL function will_churn that uses the variables the model needs. We now include the columns created by one-hot encoding in the previous section.

Now we need to set up the Aurora database to call the Amazon SageMaker endpoint and pass the data it needs to return a prediction. We do so by creating a SQL function that collects the data, performs any transformations we need (for example, combining data or changing data types), and calls the model endpoint.

Our original data contained numeric variables and several categorical variables (such as area_code and int_plan). During the creation of the ML model, the categorical variables were converted to one-hot vectors. This conversion can lead to an explosion in the number of columns the model uses; for this use case, the original 15 feature columns became 70 after conversion to one-hot. Because only a short list of the columns was used, we can build a model using only those columns. This simplifies the database call to the ML model because it reduces the number of fields the SQL needs to include in the call.

In the final model, we used only one of the categorical values: int_plan_no. However, the database column is int_plan, and must be converted to its one-hot representation. There are two ways to approach this problem: add transformation code to the ML endpoint or create functions in the SQL database that represent one-hot encoded variables.

For simplicity, this post demonstrates the second option. We create a SQL function, IntPlanNo(int_plan), that returns the one-hot encoded value for this field. For more complex transformations, the first option is likely a better option.

Ready, set, go!

Now, finally, we can put all the pieces together!

We’ve taken the incentive programs defined by Marketing and, for demonstration purposes, created a Python function, assess_and_recommend_incentive. The function takes the customer phone number and message the customer sent to customer service (via a chat, for example). It queries the database for customer details needed by Customer Service, and returns a likelihood of customer churn and a suggested incentive. The function contains several fixed incentives for customers with certain characteristics. We can also test for alternatives with a randomized incentive. The output contains the recommended incentive and a short message explaining the reason that incentive was chosen. For a production setting we’ll make these same calls from a business application.

The following code shows the results of two calls to the function. The first, with a negative customer message and a customer likely to churn, results in a credit recommendation. The second, with a neutral customer message and a less-likely-to-churn customer, results in no incentive offered.

print(assess_and_recommend_incentive( 415, '358-1921', "You morons! You charged me extra again!"))
Sentiment NEGATIVE and will_churn>0.8: $25 credit print(assess_and_recommend_incentive(408, '375-9999', "How do I dial Morocco?"))
NOT (cust_service_calls > 4 and not int_plan_no): No incentive.

Now we can give customer service the tool they need. With the customer’s phone number and text of the customer’s concern, we can immediately identify an incentive to offer based on their history in the database and their current interaction.

We can also test the effect of different values for the incentive program against the true and false positives and negatives returned by the model. This allows us to estimate the economic impact of our planned incentives.

Cleaning up

To avoid additional charges, you should delete the CloudFormation stack after walking through the solution.

You should also delete the S3 bucket and Amazon SageMaker endpoints created during model training. A cell at the end of the Part 2 notebook has code that deletes these for you.


Now we have the ability to detect, during a customer interaction, whether this is an at-risk customer, and to intervene by providing them with an incentive to stay. We can choose the factors on which we want to base our response: the sentiment of this interaction, combined with the customer’s current characteristics, interaction history, and estimated risk. We can come back to our CEO with a proposed set of incentives, and an estimate for the cost of our incentive program; or even with a set of alternatives, showing separate optimization curves for each. We’ve gained valuable insights into our customers’ behavior, and we have plans to gather more data for more insights.

To gather more data on effective incentives and customer behavior, we can also randomize our responses. We have the beginnings of understanding the key factors, and the beginnings of an experimentation platform. We can use that data in a future ML model to further refine the incentives offered.

We can now begin to add experimentation and nuance to our incentive program, such as the following:

  • Record the customer’s sentiment when we offer an incentive, and also their churn prediction at this time, the current value of key drivers, and the incentive offered.
  • Randomize the incentives offered to perform A/B testing of different incentives to customers with similar characteristics.
  • Experiment with different kinds of incentives beyond a simple dollar rebate, such as new telephones, plan upgrades, and more.
  • Store the incentive offered and later analyze their use and the results achieved. How much of an incentive must we offer to which kinds of customers to reduce churn? How much is retaining that customer worth to us, versus the cost of the incentive? What is the customer’s sentiment on hearing the incentive?
  • Add economic analysis. How much is it worth to keep this customer? What’s the optimal threshold score to maximize profit?

As we explore these alternatives, we move from prediction to providing tangible and actionable business value. Over to you, Marketing!

About the authors

Dr. Veronika Megler is Principal Consultant, Data Science, Big Data & Analytics, for AWS Professional Services. She holds a PhD in Computer Science, with a focus on scientific data search. She specializes in technology adoption, helping companies use new technologies to solve new problems and to solve old problems more efficiently and effectively.

Vitalina Komashko is a Data Scientist for AWS Professional Services. She holds a PhD in Pharmacology and Toxicology, but don’t hold it against her. She specializes in reproducible research, clean code and helping biotech and pharma to formulate and solve problems with scalable solutions.



Clearview AI sued by ACLU for scraping billions of selfies from social media to power its facial-recog-for-cops system



The American Civil Liberties Union has sued Clearview AI for scraping billions of photos from public social media profiles, without people’s explicit consent, to train its facial-recognition system.

The lawsuit [PDF], filed on Thursday at the Circuit Court of Cook County, Illinois, claims Clearview violated the state’s stringent Biometric Information Privacy Act (BIPA). Companies operating in Illinois must obtain explicit consent from individuals if they collect their biometric data, whether it’s in the form of fingerprints or photographs.

“Clearview has violated and continues to violate the BIPA rights of Plaintiffs’ members, clients, and program participants and other Illinois residents at staggering scale,” the lawsuit, brought by a group led by the ACLU, claimed.

“Using face recognition technology, Clearview has captured more than three billion faceprints from images available online, all without the knowledge – much less the consent – of those pictured.”

The startup, based in New York, made headlines in January when it was revealed to have amassed a database of three billion images by downloading people’s pictures from public pages on sites like Facebook, YouTube, Venmo, Instagram, and Twitter.

The dataset was used to train facial recognition algorithms, so that when images, say from a CCTV camera, are fed into Clearview’s system, the code looks for a match, and if one is found, it spits out everything it knows about that person: their harvested photos, and the URLs to the source pages that typically contain more personal information, such as names and contact details. This allows Clearview’s customers to turn faces in security camera footage stills into complete personal profiles, for example.

Initially, CEO Hoan Thon That said his upstart’s software was only intended for cops and government agents. But a hacker broke into Clearview’s systems and revealed its customer list, which contained US household staples such as Macy’s, Walmart, Wells Fargo, and Bank of America, and some universities.

The unregulated use of the technology has prompted many other groups to file the lawsuit against Clearview alongside the ACLU, including other non-profits and social justice organizations that support sex workers and the Latino population in Illinois.

Illustration of facial recognition

Hacker swipes customer list from controversial face-recog-for-Feds Clearview. Its reaction? ‘A part of life’


“Given the immutability of our biometric information and the difficulty of completely hiding our faces in public, face recognition poses severe risks to our security and privacy,” the ACLU said in its lawsuit.

“The capture and storage of faceprints leaves people vulnerable to data breaches and identity theft. It can also lead to unwanted tracking and invasive surveillance by making it possible to instantaneously identify everyone at a protest or political rally, a house of worship, a domestic violence shelter, an Alcoholics Anonymous meeting, and more.

“And, because the common link is an individual’s face, a faceprint can also be used to aggregate countless additional facts about them, gathered from social media and professional profiles, photos posted by others, and government IDs.”

Tech companies have also tried to thwart Clearview’s slurping of photos. In February, Google, YouTube, Twitter, and Facebook all served the startup cease-and-desist letters ordering it to stop stealing images from their platforms, and to delete existing pics in its massive database.

“For far too long tech companies have misused our most sensitive data while facing too little consequence,” said Abraham Scarr, director at the Illinois Public Interest Research Group, a nonprofit organization that’s also suing Clearview alongside the ACLU.

“The BIPA is unique in that it allows Illinois residents to control not only their biometric information, but also the laws governing its use, putting the power back into the hands of the people.”

Clearview’s lawyer Tor Ekeland told The Register: “Clearview AI is a search engine that uses only publicly available images accessible on the internet. It is absurd that the ACLU wants to censor which search engines people can use to access public information on the internet. The First Amendment forbids this.” ®

Sponsored: How to simplify data protection on Amazon Web Services


Continue Reading


AI Autonomous Cars And The Problem Of Where To Drop Off Riders



Having the AI self-driving car know where to drop off human passengers is a puzzling problem not high on the priority list of developers. (GETTY IMAGES)

By Lance Eliot, the AI Trends Insider

Determining where to best drop-off a passenger can be a problematic issue.

It seems relatively common and downright unnerving that oftentimes a ridesharing service or taxi unceremoniously opts to drop you off at a spot that is poorly chosen and raft with complications.

I remember one time, while in New York City, a cab driver was taking me to my hotel after my having arrived past midnight at the airport, and for reasons I’ll never know he opted to drop me about a block away from the hotel, doing so at a darkened corner, marked with graffiti, and looking quite like a warzone.

I walked nearly a city block at nighttime, in an area that I later discovered was infamous for being dangerous, including muggings and other unsavory acts.

In one sense, when we are dropped off from a ridesharing service or its equivalent, we often tend to assume that the driver has identified a suitable place to do the drop-off.

Presumably, we expect as a minimum:

·         The drop-off is near to the desired destination

·         The drop-off should be relatively easy to get out of the vehicle at the drop-off spot

·         The drop-off should be in a safe position to get out of the vehicle without harm

·         And it is a vital part of the journey and counts as much as the initial pick-up and the drive itself.

In my experience, the drop-off often seems to be a time for the driver to get rid of a passenger and in fact the driver’s mindset is often on where their next fare will be, since they’ve now exhausted the value of the existing passenger and are seeking more revenue by thinking about their next passenger.

Of course, you can even undermine yourself when it comes to doing a drop-off.

The other day, it was reported in the news that a woman got out of her car on the 405 freeway in Los Angeles when her car had stalled, and regrettably, horrifically, another car rammed into her and her stalled vehicle. A cascading series of car crashes then occurred, closing down much of the freeway in that area and backing up traffic for miles.

In some cases, when driving a car ourselves, we make judgements about when to get out of the vehicle, and in other cases such as ridesharing or taking a taxi, we are having someone else make a judgement for us.

In the case of a ridesharing or taxi driver, I eventually figured out that as the customer I need to double-check the drop-off, along with requesting an alternative spot to be dropped off if the circumstances seem to warrant it. You usually assume that the local driver you are relying on has a better sense as to what is suitable for a drop-off, but the driver might not be thinking about the conditions you face and instead could be concentrating on other matters entirely.

Here’s a question for you, how will AI-based true self-driving driverless autonomous cars know where to drop-off human passengers?

This is actually a quite puzzling problem that though not yet seemingly very high on the priority list of AI developers for autonomous cars, ultimately the drop-off matter will rear its problematic head as something needing to be solved.

For my overall framework about autonomous cars, see this link:

For why achieving a true self-driving car is like a moonshot, see my explanation here:

For my indication about edge or corner cases in AI autonomous cars, see this link:

For dangers that await pedestrians and how AI self-driving car should respond, see my discussion here:

AI Issues Of Choosing Drop-off Points

The simplistic view of how the AI should drop you off consists of the AI system merely stopping at the exact location of where you’ve requested to go, as though it is merely a mathematically specified latitude and longitude, and then it is up to you to get out of the self-driving car.

This might mean that the autonomous car is double-parked, though if this is an illegal traffic act then it goes against the belief that self-driving cars should not be breaking the law.

I’ve spoken and written extensively that it is a falsehood to think that autonomous cars will always strictly obey all traffic laws, since there are many situations in which we as humans bend or at times violate the strict letter of the traffic laws, doing so because of the necessity of the moment or even at times are allowed to do so.

In any case, my point is that the AI system in this simplistic perspective is not doing what we would overall hope or expect a human driver to do when identifying a drop-off spot, which as I mentioned earlier should have these kinds of characteristics:

·         Close to the desired destination

·         Stopping at a spot that allows for getting out of the car

·         Ensuring the safety of the disembarking passengers

·         Ensuring the safety of the car in its stopped posture

·         Not marring the traffic during its stop

·         Etc.

Imagine for a moment what the AI would need to do to derive a drop-off spot based on those kinds of salient criteria.

The sensors of the self-driving car, such as the cameras, radar, ultrasonic, LIDAR, and other devices would need to be able to collect data in real-time about the surroundings of the destination, once the self-driving car has gotten near to that point, and then the AI needs to figure out where to bring the car to a halt and allow for the disembarking of the passengers. The AI needs to assess what is close to the destination, what might be an unsafe spot to stop, what is the status of traffic that’s behind the driverless car, and so on.

Let’s also toss other variables into the mix.

Suppose it is nighttime, does the drop-off selection change versus when dropping off in daylight (often, the answer is yes). Is it raining or snowing, and if so, does that impact the drop-off choice (usually, yes)? Is there any road repair taking place near to the destination and does that impact the options for doing the drop-off (yes)?

If you are saying to yourself that the passenger ought to take fate into their own hands and tell the AI system where to drop them off, yes, some AI developers are incorporating Natural Language Processing (NLP) that can interact with the passengers for such situations, though this does not entirely solve this drop-off problem.


Because the passenger might not know what is a good place to drop-off.

I’ve had situations whereby I argued with a ridesharing driver or cabbie about where I thought I should be dropped-off, yet it turned out their local knowledge was more attuned to what was a prudent and safer place to do so.

Plus, in the case of autonomous cars, keep in mind that the passengers in the driverless car might be all children and no adults. This means that you are potentially going to have a child trying to decide what is the right place to be dropped off.

I shudder to think if we are really going to have an AI system that lacks any semblance of common-sense be taking strict orders from a young child, whereas an adult human driver would be able to counteract any naïve and dangerous choice of drop-offs (presumably, hopefully).

For the use of Natural Language Processing in socio-conversations, see my discussion here:

For my explanation about why it is that AI self-driving cars will need to drive illegally, see this link:

For the role of children as riders in AI autonomous cars, see my indication here:

For my insights about how nighttime use of AI self-driving cars can be difficult, see this link:

For the role of ODD’s in autonomous cars, here’s my discussion:

More On The Drop-off Conundrum

The drop-off topic will especially come to play for self-driving cars at a Level 4, which is the level at which an autonomous car will seek to pullover or find a “minimal risk condition” setting when the AI has reached a point that it has exhausted its allowed Operational Design Domain (ODD). We are going to have passengers inside Level 4 self-driving cars that might get stranded in places that are not prudent for them, including say young children or perhaps someone elderly and having difficulty caring for their own well-being.

It has been reported that some of the initial tryouts of self-driving cars revealed that the autonomous cars got flummoxed somewhat when approaching a drop-off at a busy schoolground, which makes sense in that even as a human driver the chaotic situation of young kids running in and around cars at a school can be unnerving.

I remember when my children were youngsters how challenging it was to wade into the morass of cars coming and going at the start of school day and at the end of the school day.

One solution apparently for the reported case of the self-driving cars involved re-programming the drop- off of its elementary school aged passengers at a corner down the street from the school, thus apparently staying out of the traffic foray.

In the case of my own children, I had considered doing something similar, but subsequently realized that it meant they had a longer distance to walk to school, providing other potential untoward aspects and that it made more sense to dig into the traffic and drop them as closely to the school entrance as I could get.

Some hope that Machine Learning and Deep Learning will gradually improve the AI driving systems as to where to drop off people, potentially learning over time where to do so, though I caution that this is not a slam-dunk notion (partially due to the lack of common-sense reasoning for AI today).

Others say that we’ll just all have to adjust to the primitive AI systems and have all restaurants, stores, and other locales all stipulate a designated drop-off zone.

This seems like an arduous logistics aspect that would be unlikely for all possible drop-off situations. Another akin approach involves using V2V (vehicle-to-vehicle) electronic communications, allowing a car that has found a drop-off spot to inform other nearing cars as to where the drop-off is. Once again, this has various trade-offs and is not a cure-all.


It might seem like a ridiculous topic to some, the idea of worrying about dropping off people from autonomous cars just smacks of being an overkill kind of matter.

Just get to the desired destination via whatever coordinates are available, and make sure the autonomous car doesn’t hit anything or anyone while getting there.

The thing is, the last step, getting out of an autonomous car, might ruin your day, or worse lose a life, and we need to consider holistically the entire passenger journey from start to finish, including where to drop-off the humans riding in self-driving driverless cars.

It will be one small step for mankind, and one giant leap for AI autonomous cars.

Copyright 2020 Dr. Lance Eliot

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column:]


Continue Reading


AI Careers: Kesha Williams, Software Engineer, Continues Her Exploration



Helping information technology to diversify and especially to help women of color achieve in technology and business, has been a personal goal for Kesha Williams, software engineer, author and speaker. (GETTY IMAGES)

By John P. Desmond, AI Trends Editor

We recently had a chance to catch up on the career of Kesha Williams, software engineer, author, speaker and instructor. AI Trends published an Executive Interview with Kesha in June 2018. At the time she was in the Information Technology department at Chick-fil-A, the restaurant chain, with responsibility to lead and mentor junior software engineers, and deliver on innovative technology.

She decided to move on from Chick-fil-A after 15 years in June 2019. Now she works at A Cloud Guru, an online education platform for people interested in cloud computing. Most of the courses prepare students for certification exams. The company was established in Melbourne, Australia in 2015.

“I wanted a role that allowed me to be more hands on with the latest, greatest technology,” she said in a recent interview. “And I wanted to be able to help people on a broader scale, on a more global level. I always felt my part of being here on the planet is to help others, and more specifically to help those in tech.”

Kesha Williams, software engineer, author, speaker and instructor

A Cloud Guru offers certifications for Amazon Web Services (AWS), Microsoft Azure and Google Cloud. It also has what Williams calls “cloud adjacent” courses including on Python programming and machine learning. “These courses will help you ‘skill up’ in the cloud and prepare for certification exams,” she said.

Kesha’s role is as a training architect, focusing on online content around AWS, specifically in the AI space. “Many people have taken this time being at home, to work on skills or learn something new. It’s a great way to spend time during the lockdown,” she advised. A true techie.

AWS DeepComposer Helps Teach About Generative AI and GANs

Most recently, she has been using AWS DeepComposer, an educational training service through AWS that allows the user to compose music using generative AI and GANs (generative adversarial networks, a class of machine learning frameworks). “I have been learning about that, so I can teach others about machine learning and music composition,” she said.

Using music samples, the user trains a music genre model. That model learns how to create new music, based on studying the music files you upload to it. The user plays a melody on a keyboard, gives it to the model, the model composes a new song by adding instruments. She is working on a web series to teach students about that process.

“It’s a fun way to teach some of the more complex topics of GANs and machine learning,” she said. Fortunately she can fall back on youth choir days playing the piano. “I’m remembering things,” she said.

Amazon makes it easy to start out, not charging anything for up to 500 songs. A student can buy the keyboard for $99, or use a virtual keyboard available on the site. Behind the scenes, Amazon SageMaker is working. That will cost some money if the student continues. (SageMaker is a cloud machine-learning platform, launched in November 2017. It enables developers to create, train and deploy machine-learning models in the cloud, or on edge devices.)

So far, Williams has done about 30 songs. “I have used my machine learning skills to train my own genre model. I trained a reggae model; I love reggae.”

Kesha’s Korner is a blog on A Cloud Guru where Williams introduces people to machine learning, offering four to six-minute videos on specific topics. The videos are free to watch; pricing for the A Cloud Guru courses come with membership priced from $32/mo to $49/mo depending, “It’s been a fun series to demystify machine learning,” she said. “It generates a lot of conversations. I often receive feedback from students on which topics to talk about.”

Woman Who Code Planning Virtual Conference

Women Who Code is another interest. The organization works to help women be represented as technical leaders, executives, founders, venture capitalists, board members and software engineers.

The Connect Digital 2020 is the organization’s first entirely virtual conference, to be held on three successive Fridays in June, with Williams scheduled for Friday, June 19. At that meeting, she will deliver a talk about using machine learning for social good, then kick off a “hackathon” to start the following week. The hackathon will start with three technical workshops, the first an introduction to machine learning tools, the second about preparing data, the third about building models. “Their challenge is to take everything they have learned and use machine learning to build a model to help battle the spread of the Covid-19 virus,” she said. “They will have a month to go off and build it, then present it to a panel of judges.” The winner receives a year of free access to the A Cloud Guru platform.

“There are a lot of software engineers that want to make a transition to data science and machine learning,” she said.

Asked what advice she would have for young people or early-career people interested in exploiting AI, Williams said, “Whenever I try to demystify machine learning for people, I tell them it’s complex, but not as complex as most people make it out to be. I thought at first you needed a PhD and to work in a research lab to grasp it. But there are many tools and services out there, especially from AWS, that make these complex technologies approachable and affordable to play around with.

“When you are first learning, you will make a lot of mistakes,” she said. “Don’t beat yourself up. Just stay at it.”

Williams has concerns about AI going forward. “I have always been concerned about the lack of diversity in AI, about the bias issues and the horror stories we have seen when it comes to certain bad-performing models that are used to make decisions about people. It’s still an issue; we need to continue to talk about it and solve it.”

Being in information technology for 25 years has been and continues to be a good career. “It’s still exciting for me. Every day there is something new to learn.”

Learn more at Kesha’s Korner and Women Who Code.


Continue Reading
Blockchain11 mins ago

Bitcoin Price Prediction: BTC/USD Stabilizes Above $9,500 As The Bulls Struggling To Conquer $9,800

Blockchain14 mins ago

Moderna (MRNA) Shares Up 0.37% amid Investors’ Concerns about Stock Sales

Cannabis22 mins ago

Louisiana OKs medical marijuana reform, but minimal market boost seen

Blockchain23 mins ago

Twitter’s Bitcoin Sentiment Suggests a Price Breakout is Imminent: Here’s Why

Blockchain49 mins ago

Tether Integrates with OMG Sidechain to Decrease Load on Ethereum

Blockchain58 mins ago

Chainlink, Ethereum, Cardano and DigiByte Among 27 Altcoins Outperforming Bitcoin (BTC) in 2020

Blockchain1 hour ago

How to Learn Any Skill Fast

Blockchain1 hour ago

Leading US Crypto Exchange Bittrex Global Lists WINGS

Blockchain1 hour ago

AMZN and AAPL Stocks 1% Up, TGT 2% Down as Amazon, Target and Apple Temporarily Close Stores Due to U.S. Riots

Blockchain1 hour ago

Draper Goren Holm’s LA Blockchain Summit Celebrates Going Virtual With A $1 Million Bitcoin Giveaway

Blockchain1 hour ago

Video Director of Eminem’s ‘Without Me’ Wants To Buy Bitcoin’s Dip

Blockchain1 hour ago

Following Police Brutality and Riots, Hackers of Anonymous Attack Minneapolis Police

Cannabis1 hour ago

Montana patients will be free to shop at any dispensary starting Tuesday

Blockchain1 hour ago

Inflation may grow as the US prevents negative interest rates, boosting Bitcoin bull case

Blockchain1 hour ago

Bitcoin Shoots Up to $9,500

Blockchain1 hour ago

Crypto Confusion: Long-Legged Doji Across Altcoin Market Hints At Trend Change

Blockchain1 hour ago

Anonymous Does its Part to Demand Justice for George Floyd

Blockchain1 hour ago

FXSpotStream Reports Slight Rebound in May Volumes

Blockchain2 hours ago

Vitalik Claims that ‘Ethereum’s Layer 2 Scaling Strategy Has *Basically* Succeeded’

Cannabis2 hours ago

MJardin aborts acquisition of Nevada cannabis edibles maker