Connect with us

Published

on

# Beginners Guide to Debugging TensorFlow Models

If you are new to working with a deep learning framework, such as TensorFlow, there are a variety of typical errors beginners face when building and training models. Here, we explore and solve some of the most common errors to help you develop a better intuition for debugging in TensorFlow.

Photo by Dmitriy Demidov on Unsplash.

TensorFlow is one of the most famous deep learning models, and it is easy to learn. This article will discuss the most common errors a beginner can face while learning TensorFlow, the reasons, and how to solve these errors. We will discuss the solutions and also what experts from StackOverflow say about them.

### Example 1: Wrong Input Shape for CNN layer

Suppose you are making a Convolutional Neural Network, now if you are aware of the theory of CNN, you must know that a CNN (2D) takes in a complete image as its input shape. And a complete image has 3 color channels that are red, green, black. So the shape of a normal image would be (height, width, color channels). But if you pass in a grayscale image, it is normally (height, width), and the color channel is excluded as shown in the code.

```model = Sequential([ Conv2D(32, 5, input_shape=(28,28), activation=’relu’), Flatten(), Dense(10, activation=’softmax’)
]) ```

Now, if you train this model, you would get an error.

`ValueError: Input 0 of layer conv2d_1 is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: (None, 28, 28) `

This is because we are passing the input shape of (28,28) and 1 extra dimension added by TensorFlow for Batch size, so the error message says that it found ndim=3, but the CNN has expected min_ndim=4, 3 for the image size and 1 for the batch size.

So you can solve this error by changing the input shape in the first CNN layer and reshaping your inputs before passing it to CNN.

`X_train=x_train.reshape(number_of_rows,28,28,1) `

This will change your input from (number_of_rows, height, width) to (number_of_rows, height, width, color_channel) where color_channel is equal to 1, showing that it is a grayscale image. Now your CNN is ready to work. You can check this StackOverflow question for more details.

### Example 2: Negative Dimension Size

This is one of the common errors that new practitioners deal with when playing with CNNs or other models that change input shape after each layer. Now the output shape of CNN depends on several factors such as the number of filters, kernel size, padding type, and stride size. Let’s say you have a model:

```model = Sequential([ Conv2D(32, 5, input_shape=(28,28,1), activation='relu'), Conv2D(32, 5, activation='relu'), Conv2D(32, 5, activation='relu'), Conv2D(32, 5, activation='relu'), Conv2D(32, 5, activation='relu'), Conv2D(32, 5, activation='relu'),
]) model.summary() ```

You can see that the input size is getting smaller and smaller, and if you add any more CNN layers, it will reduce to negative and hence raise a negative dimension error. So you need to understand how tuning different CNN parameters will affect your output shape. The error message after adding another CNN layer would be a long traceback message, but you need to find the right part of it, that is:

`ValueError: Negative dimension size caused by subtracting 5 from 4 for ‘{{node conv2d_16/Conv2D}} = Conv2D[T=DT_FLOAT, data_format=”NHWC”, dilations=[1, 1, 1, 1], explicit_paddings=[], padding=”VALID”, strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, conv2d_16/Conv2D/ReadVariableOp)’ with input shapes: [?,4,4,32], [5,5,32,32]. `

There are different solutions to it, such as changing the padding or stride size, changing the number of layers, and tuning the other hyperparameters while keeping the output shape of that layer in your mind. You can have a look at a detailed discussion on it on StackOverflow here and here, where several good solutions with reasons are offered.

### Example # 3: Wrong Output Shape

This is also a common error that beginners face of having the wrong number of nodes in the last layer. In the last layer of any neural network, you need to have the number of nodes equal to the number of classes you have or the number of outputs you want. For example, in a regression task, you normally have 1 node in the output layer because you need a single continuous value as output. In the classification task, you have n number of nodes in the output layers, which is equal to the total number of unique classes.

Let’s say you have 10 unique classes in your example, and you specify 9 in your output layer as follows:

```model = Sequential([ Conv2D(32, 5, input_shape=(28,28,1), activation=’relu’), MaxPool2D((2,2)), Conv2D(32,3, activation=’relu’), Flatten(), Dense(9, activation=’softmax’)
]) ```

Now when you train your model after compiling it, it will raise an error message.

`model.compile(‘adam’,’categorical_crossentropy’, [‘acc’]) model.fit(X_train, y_train, epochs=3) `

Error Message:

`ValueError: Shapes (32, 10) and (32, 9) are incompatible `

And you can see that as the error message says, shapes are not compatible. You can have a look at this thread in StackOverflow for more details regarding this error.

### Example # 4: Unknown Loss function

This error, as the name shows, generates when you have used a loss function that does not exist in Tensorflow.

Let’s say you compile a model

```model.compile('adam','sparse_categorical_crossentropy', ['acc'])
```

Now you have used the sparse_categorical_crossentropy loss function here, which does not exist due to wrong spelling, and a lot of beginners make similar spelling mistakes. Now the tricky part is that you will not get the error message on the compilation. In fact, you will get the error message when you fit the model.

`model.fit(X_train, y_train, epochs=1) `

Now you will receive the long error message, which you can trace and find useful information from it.

```ValueError Traceback (most recent call last) — -> 11 model.fit(X_train, y_train, epochs=1)
#Long Error traceback
ValueError: Unknown loss function: sparse_categoricalcrossentropy ```

### Example #5 Shape not compatible with appropriate function

This is a very common error message, and it appears when you are using a function or a layer or anything similar, and it expects a specific type of shape, but the shape you are passing is different from the required shape. All of these will generate a related or similar error message that is ValueError: Shape mismatch:

Let’s see an example.

Suppose your output labels are in one-hot-matrix format.

Credits: Medium.

Now, these output labels require a special loss function for simple classification tasks that is categorical_crossentropy, and if you pass in sparse_categorcial_crossentropy that is used for Label Encoded Output labels, this will generate a shape-mismatch error.

```model = Sequential([ Conv2D(32, 5, input_shape=(28,28,1), activation=’relu’), MaxPool2D((2,2)), Conv2D(32,3, activation=’relu’), Flatten(), Dense(10, activation=’softmax’)
])
model.fit(X_train, y_train, epochs=1) ```

Since our output y_train is in the one-hot matrix format, it will generate an error.

`ValueError: Shape mismatch: The shape of labels (received (320,)) should equal the shape of logits except for the last dimension (received (32, 10)). `

This error means that the sparse categorical cross-entropy loss function expects a single vector, and we are passing a matrix.

### Example # 6 Wrong Loss function

This is not an error, but rather a mistake where your model’s performance would not be improving and giving very bad results. While there can be many different reasons for it, a common reason behind it is using the wrong loss function. For example, in classification tasks, you are supposed to use the cross-entropy or related loss function, and if you are using a loss function that is not suitable for classification tasks, then your model will not improve.

```model = Sequential([ Conv2D(32, 5, input_shape=(28,28,1), activation='relu'), MaxPool2D((2,2)), Conv2D(32,3, activation='relu'), Flatten(), Dense(10)
])
model.compile('rmsprop','mae', ['accuracy'])
model.fit(X_train, y_train, epochs=3) ```

Here we are using the mean absolute error loss function instead of the cross-entropy loss function, and as a result of this, our model is not performing.

You can see the accuracy stuck at 10% and loss is also not improving. If you are stuck at a similar stage where your model is not improving, I suggest you take a look at every single step back where you have specified things and think upon what can be the possible problem due to which my model is not training correctly, because you are not going to see any error message in this case. Also, you can ask questions at any community, such as r/learnmachinelearning on Reddit or StackOverflow.

Related:

# Robinhood CEO says he is considering offering U.S. retirement accounts

Published

on

By Echo Wang and Krystal Hu

(Reuters) – Robinhood Markets Inc is considering launching U.S. retirement accounts, CEO and co-founder Vlad Tenev said on Saturday in a webcast with users of its trading app looking to participate in its initial public offering, which is set to price next week.

The online brokerage has about 18 million funded investment accounts on its platform, most of which are held by retail traders.

Offering individual retirement accounts (IRAs) and Roth IRAs, which offer tax advantages to those saving for retirement, would allow Robinhood to tap a vast market. Americans held \$12.6 trillion in IRAs at the end of March, up 2.8% from the end of December, according to the Investment Company Institute.

“We are interested in building more account types, including IRAs and Roth IRAs, we’ve been hearing that a lot from our customers. We want to make first-time investors into long-term investors,” Tenev said in response to an investor question.

Due to the penalties involved in withdrawing money, IRAs tend to attract long-term investments, rather than the quick flip in stocks, options and cryptocurrencies that some investors turn to Robinhood for.

In his webcast, however, Tenev said: “We see evidence that the majority of our customers are primarily buy and hold.”

Robinhood, which is targeting a valuation of up to \$35 billion in its IPO, has said it will allocate 20% to 35% of shares offered to its users, an unusual move for a high-profile offering. One of the reasons many IPOs enjoy a first-day trading pop is because the retail investors that Robinhood has invited are excluded and must buy shares in the open market.

Robinhood launched its IPO Access platform earlier this year to enable users to buy into the IPOs of other companies if it can negotiate deals with the investment banks handling them.

Some individual investors are calling for a boycott of Robinhood’s IPO on Reddit and other social media over its handling of the ‘meme’ stock-trading frenzy in January. Robinhood placed restrictions on buying GameStop Corp and other stocks that hedge funds had bet against, on grounds it was needed for the financial and operational stability of its platform.

Tenev said in Saturday’s webcast that Robinhood had invested in the stability of its platform to avoid another such incident.

PAYMENT FOR ORDER FLOW

Robinhood’s popularity has soared over the past 18 months of coronavirus-induced social restrictions that have kept many retail investors at home. It has said its mission is to “democratize finance for all” by allowing users to make unlimited commission-free trades in stocks, exchange-traded funds, options and cryptocurrencies.

The brokerage has been criticized for relying on “payment for order flow” for most of its revenue, under which it receives fees from market makers for routing trades to them and does not charge users for individual trades, however.

Critics argue the practice, which is used by many other brokers, creates a conflict of interest, on the grounds that it incentivizes brokers to send orders to whoever pays the higher fees. Robinhood contends that it routes trades based on what is cheapest for its users, and that charging a commission would be more expensive.

Robinhood chief financial officer Jason Warnick left the door open for the company to change the practice if necessary.

“If a ban or other limitations on it were to be imposed, we believe Robinhood and the industry would adapt and explore other revenue sources,” Warnick said.

Robinhood was founded in 2013 by Stanford University roommates Tenev and Baiju Bhatt, who will hold nearly two-thirds of the voting power after the offering, a filing with the stock exchange showed.

Robinhood customer Minjie Xu, who works as a software engineer in Missouri, remained unimpressed after the presentation on concerns the offering was overpriced.

“This is not unique to them, as I think most IPOs are overpriced,” Xu told Reuters.

(Reporting by Echo Wang and Krystal Hu in New York; Editing by Greg Roumeliotis and Sonya Hepinstall)

##### Image Credit: Reuters

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.

# Manu Smadja: CEO at MPOWER Financing, a Fintech Firm Facilitating Educational Loans, Shares Business Strategy

Published

on

MPOWER Financing, a global education loan provider, recently secured an equity investment of \$100 million. The new funding is in addition to the \$60 million MPOWER raised earlier this year from Tilden Park Capital Management that also participated in the current round along with ETS Strategic Capital.

Other investors were King Street Capital Management, Drakes Landing Associates, and Pennington Alternative Income Management.

We recently connected with Manu Smadja, CEO at MPOWER Financing, which is a Public Benefit Corporation whose stated mission is “to reduce the financial barriers to international education.” The company is based in Washington, DC, with business offices in New York and Bangalore, India.

The firm was established by Manu Smadja and Michael Davis back in 2014, both of whom attended INSEAD Paris where they first met each other. Our conversation with Manu Smadja is shared below.

### How did you identify this as a key requirement, and what are some of the long-term benefits that your organization has been able to provide?

Manu Smadja: MPOWER Financing’s mission is to remove the financial barriers that many high-promise students from around the world encounter attending school overseas. International and DACA students simply do not have access to the types of programs that American students can take advantage of, including federal student loan subsidies, grants, national scholarships, etc. Additionally, they are often excluded from many private loan options available due to a lack of domestic credit history.

We apply the latest technology and processes to fulfill this mission, ensuring we’re able to help as many students as we can as seamlessly as possible. This includes looking at a variety of different data sources along with projecting future earning potential to determine an applicant’s credit risk long-term.

We also understand that funding is only part of the equation. We provide additional career guidance and resources through our Path2Success program, which helps our students prepare for the unique challenges they may face when it comes to entering the workforce. This, in turn, yields significantly better credit outcomes and lower-than-average loan default rates.

We are constantly rethinking how we use technology to transform our business. We’re currently exploring machine learning to help identify trends and patterns in customer support, which will help us better address students’ questions and challenges.

### Crowdfund Insider: Your website states that people can “fund their dreams on their own.” Candidates can apply for your co-signer loans and scholarships to study in the U.S. and Canada. How does your Fintech platform make this possible?

Manu Smadja: Unlike traditional financing options available to international and DACA students (usually brick and mortar banks in the student’s home country), MPOWER loans do not require a cosigner or collateral for approval. Our approval process is also not based on credit history alone. Instead, we have built a proprietary algorithm that factors in future earning potential and analyzes disparate data sources, such as employment history and credit history domestically and abroad.

This model ensures that high-promise students have access to education without putting an undue financial burden on their families. MPOWER’s loans are approved instantly and are granted at competitive interest rates.

By amalgamating and evaluating the many factors that will contribute to their success and solvency, we make it possible for high-promise students who are left out of the traditional lending process to fund their endeavors in higher education in the U.S. and Canada.

### How can Fintech enhance this process from the perspective of learners and the organizations that provide education?

Manu Smadja: Our global financial systems have largely not kept up with the pace of innovation seen in other sectors of the consumer economy. While online applications and accessibility may be more commonplace, particularly due to the pandemic, the back-end systems that support these technologies are cumbersome and often outdated. Typically, the first option for many international students to fund their education is through traditional brick and mortar banks in their home countries. Not only can the lending process take longer, the majority of these loan providers are also still using conventional credit mechanisms to determine eligibility.

Since our founding, we have been committed to rethinking and overhauling how international students fund their education. This includes removing prohibitive barriers, such as requiring collateral or having a cosigner. Thanks to data and our proprietary algorithm, we gain a more detailed and comprehensive financial picture of each applicant to determine if they’re a good candidate for a loan.

Naturally, our model breaks down barriers to funding, while ensuring that each transaction is financially sound and does not present a credit risk to our business.

### How can financial technology be leveraged to expedite the loan application review process? Does your software use any A.I. or machine learning algorithms to improve the overall process?

Manu Smadja: A faster approval and processing pathway is something we strive for, but the process must be anchored by accuracy. We recently launched our Instant Offer product, enabling a student to check their eligibility in 30 seconds and apply online in less than 30 minutes. Based on the information we collect; we are able to provide a conditional loan offer in real-time.

### Crowdfund Insider: What are your plans for this year and beyond? What technological improvements do you plan to incorporate into your existing platform?

Manu Smadja: In 2021, MPOWER has raised a total of \$160 million including the \$100 million in equity on July 13. These funds are being used to scale and automate our operations, enabling us to help more students.

While many companies would use an investment like this to expand product offerings, we want to stay true to our core offering: lending for high-promise students and tools and education to help prepare students to enter the workforce. This means investing in our team and technology.

### Crowdfund Insider: Earlier this year, MPOWER partnered with Sallie Mae to expand higher education access, specifically for international and DACA students.

Manu Smadja: We are always looking for organizations to partner with, particularly when it builds the bridge between funding and a students’ dreams to pursue higher education. The partnership with Sallie Mae helps fill the gaps found in traditional lending services, offering credit-based private loan products that students can apply for with an eligible U.S. citizen or permanent resident co-signer. When a co-signer is unavailable, MPOWER works with Sallie Mae and leverages its proprietary data and technology to underwrite high-promise international students.

We look forward to partnering with more lenders so that we can help more students fulfill their educational goals.

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.

# Implementing LSTM for Human Activity Recognition using Smartphone Accelerometer data

Published

on

Implementing LSTM for Human Activity Recognition

PlatoAi. Web3 Reimagined. Data Intelligence Amplified.

Source: https://www.analyticsvidhya.com/blog/2021/07/implementing-lstm-for-human-activity-recognition-using-smartphone-accelerometer-data/

# Plunging into Deep Learning carrying a red wine

Published

on

Deep Learning |Plunging into Deep Learning carrying a red wine