## Neural networks

A neural network is based somewhat on a real brain - the idea being that data comes into a bunch of neurons at the front, gets propagated through multiple layers, and an answer comes out the end. We've covered this a little in another blog post, so here we'll just focus on what goes on under the covers.

## Math time

A neural network is a type of non-linear function that can approximate any other function. How well it can approximate is based on the number of links it has between its layers of neurons. In other words, more neurons and more layers gives you a better (but slower) neural network. If you want to have a play with this idea, have a look at the Tensorflow Playground - if you can't get it to match, try a mix of more layers and more neurons per layer.

## More math time

We have a non-linear function, and we need to approximate something based on stochastic gradient descent. In other words, we get a bunch of samples as input, and we need to adjust the network based on what the output *should* be, and then the network will change to give us answers that are slightly closer to what we expect. If our training data is a diverse sample of our real data, then we can train a neural network on our training data, and it'll also work as well with other data that it hasn't seen before.

This does seem kind of like magic though, right? What's going on under the covers?

## The calculus bit

We generally use a method called Stochastic Gradient Descent, or a method related to it. The "stochastic" part means we randomly choose samples from our training data so that the model doesn't get biased, and all the little nudges we give it start to shape it in the way we want it to behave. The "gradient descent" part means for each sample, we look at how wrong the network currently is (the "error"), and calculate a gradient (or slope, if you like), and nudge the network in that direction.

This approach is robust and it works for a lot of things, but the hard part with a neural network is trying to find the gradient for every single weight in the network - if your network is 10 layers deep (a small network by modern standards), how does an error at the output adjust the weights 10 layers back at the start?

## Backpropagation

We use partial derivatives to find the gradient at each layer, and then use that gradient to adjust the network. At each neuron, we take the sum of all of the downstream gradients (they get "backpropagated" to us), multiplied by the derivative of the activation function applied to the forward-propagated data. We then take this number, multiplied by the weight of the previous link, and add a fraction of this (multiplied by the training rate) to the current weight of that link. We also take this number multiplied by the forward-propagated data and backpropagate that to the previous neuron.

Simple... right?

This is some fiddly stuff if you're not currently a calculus student, and it took me a couple weeks to get my head around. You can find it all on the wikipedia page for backpropagation, and there are a couple of papers floating around the internet that explain it in different ways.

## It shouldn't be this hard

The idea of multiplying partial derivatives is a simple one, and I spent a long time thinking about how to make this accessible. The architecture that I've settled on for Lernmi has made a couple of trade-offs, but I feel it breaks out the backpropagation into bite-sized pieces, and I hope that it makes it easier to understand.

## Lernmi

I've published a neural network application in ruby, hosted here. The logic is broken up between Neurons and Links, and instead of the word "gradient", I've used the word "sensitivity" as it better translates to what we're using it for - high sensitivity means we adjust the weight more, low sensitivity means we adjust it less.

When we propagate forwards, it's fairly simple. Each Link in a layer takes the output from it's input Neuron, multiplies it by its weight, and inputs it to its output Neuron. Each Neuron sums all of its inputs, applies the activation function (the sigmoid function), and waits for the next Link to take the resulting value.

When we backpropagate, the logic is spread across the Neurons and Links. For the output layer, the sensitivity is just the error (the actual output minus the expected output). Each Link in the last layer takes the sensitivity from its output Neuron, and multiplies this by the sensitivity of the activation function - we call this the "output sensitivity". The reason we do this is because our neurons use their activation function when they propagate, so we need to find the sensitivity of that, and we multiply it by the sensitivity of the output Neuron to get the sensitivity of the Link. Perhaps a better name could be the "link sensitivity"? We'll call it the "output sensitivity" for now though.

We use the output sensitivity in two ways: when we update the weight, we multiply it by its input value (the larger our input value, the more we probably contributed to the error), and adjust the weight of this Link by that amount. We also need to send a sensitivity back to the input Neuron - we multiply the output sensitivity by our weight, so that the earlier layer knows how much this Link contributed to the error.

## Whose fault is this?

Don't be surprised if this is confusing the first time your read through. I would love to find a way to simplify this further - the principle is as simple as asking "what part of the network is to blame for the error". The math is a bit frustrating, as we're ultimately multiplying partial derivatives along the length of the network, but the goal of it is to do lots of tiny adjustments until the network is a good-enough approximation of the underlying data.

## What are we approximating though?

This is where it gets a little freaky. It turns out that everything - handwriting, faces, pictures of various different objects, they all have a mathematical approximation. With a big enough neural network, you can recognise people, animals, objects, and various different styles of handwriting and signwriting. Even in a board game like chess or go, you can make a mathematical approximation of which moves are better, to the point where you can have a computer that can play at a world class level.

This isn't the same as intelligence; the computer isn't thinking, but at the same time, what is thinking anyway? Is what we call "thinking" and "judging" just a mathematical approximation in our heads?

1. Useful models are made accessible by utilizing OLAP for anticipating, pattern investigation and so forth. They bolster clients in information investigation. data science course in pune

2. Well, The information which you posted here is very helpful & it is very useful for the needy like me.., Wonderful information you posted here. Thank you so much for helping me out to find the Data Analytics Course in MumbaiData analytics course in Mumbai

3. Nice Post...I have learn some new information.thanks for sharing.
ExcelR data analytics course in Pune | business analytics course | data scientist course in Pune

4. Such a very useful article. I have learn some new information.thanks for sharing.
data scientist course in mumbai

5. I have to search sites with relevant information on given topic and provide them to teacher our opinion and the article.
Data Analytics Course in Mumbai

6. Such a very useful article. Very interesting to read this article. I have learn some new information.thanks for sharing. ExcelR

7. Very nice blog here and thanks for post it.. Keep blogging...
ExcelR data science training

8. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.
data science course in mumbai

9. Just saying thanks will not just be sufficient, for the fantasti c lucidity in your writing. I will instantly grab your rss feed to stay informed of any updates.
data analytics course mumbai

10. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck... data science course

11. I finally found great post here.I will get back here. I just added your blog to my bookmark sites. thanks.Quality posts is the crucial to invite the visitors to visit the web page, that's what this web page is providing.

machine learning course

artificial intelligence course in mumbai

12. This is a wonderful article. I really enjoyed reading this article. Thanks for sharing such detailed information.
Data Science Course in Marathahalli
Data Science Course Training in Bangalore

13. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.

data analytics courses in mumbai

data science interview questions

data science course in mumbai

14. I have to search sites with relevant information on given topic and provide them to teacher our opinion and the article.

data analytics courses

data science interview questions

data science course in mumbai

15. Thanks for sharing a such a information on Artificial Intelligence. I am interested learning more about this topic.

16. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites!

AI course in mumbai

17. Nice Article.very impressed for this informative
ExcelR data analytics courses

18. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.

best digital marketing course mumbai

Data science Interview Questions

20. You have clarified the idea truly well. Was searching for this data from some time and fortunately I discovered your post. Searching forward for a greater amount of such useful updates from you

21. Hi, Thanks for sharing nice information...

Data science Interview Questions

Data Science Course

25. Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!

data science course

data science course

Data Science Course

data science course

29. I have to search sites with relevant information ,This is a
wonderful blog,These type of blog keeps the users interest in
the website, i am impressed. thank you.

30. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.

data science course

data analytics course in Bangalore

32. Informative post, i love reading such posts. Read my posts here
Laravel web development
Instapaper
Boostesports

data analytics course in Bangalore

34. Attend The Artificial Intelligence course From ExcelR. Practical Artificial Intelligence course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Artificial Intelligence course.
Artificial Intelligence Course

Correlation vs Covariance

36. Hey, i liked reading your article. You may go through few of my creative works here
Glitch
Exercism

Correlation vs Covariance

38. I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.

Correlation vs Covariance

39. The context has been explained really well. Looking forward to see more of such informative updates

40. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.

Simple Linear Regression

41. Attend The Course in Data Analytics From ExcelR. Practical Course in Data Analytics Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Course in Data Analytics.
Course in Data Analytics

Correlation vs Covariance
Simple linear regression

43. Wow what a Great Information about World Day its exceptionally pleasant educational post. a debt of gratitude is in order for the post.
Data Science Course in Bangalore

Correlation vs Covariance
Simple linear regression

45. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.

Simple Linear Regression

Correlation vs Covariance

46. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

Data Science In Banglore With Placements
Data Science Course In Bangalore
Data Science Training In Bangalore
Best Data Science Courses In Bangalore
Data Science Institute In Bangalore

Thank you..

47. I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you!

Simple Linear Regression

Correlation vs Covariance

48. This post is very simple to read and appreciate without leaving any details out. Great work!
learn360digitmg artificial intelligence online course

Correlation vs Covariance
Simple linear regression
data science interview questions

50. Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. best free instagram likes and followers app

data science interview questions

Correlation vs Covariance
Simple linear regression
data science interview questions

53. I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.

data science interview questions

54. this blog was really great, never seen a great blog like this before. i think im gonna share this to my friends..
data science training in guwahati

55. cool stuff you have and you keep overhaul every one of us

Simple Linear Regression

Correlation vs Covariance

Correlation vs Covariance
Simple linear regression
data science interview questions

360digitmg ai and machine learning courses

58. The Way Of Explaination Is Very Good And Nice Informative Topic You Have Choosen..

59. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science training

60. I have recently visited your blog profile. I am totally impressed by your blogging skills and knowledge.

Correlation vs Covariance
Simple linear regression
data science interview questions

62. Very educating story, saved your site for hopes to read more!
360digitmg ai courses

Simple Linear Regression

Correlation vs Covariance

Correlation vs Covariance
Simple linear regression
data science interview questions

Correlation vs Covariance
Simple linear regression
data science interview questions

66. Thanks for sharing great information!!!

67. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science online course

68. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.

data science interview questions

Correlation vs Covariance
Simple linear regression
data science interview questions

70. Attend The Artificial Intelligence course From ExcelR. Practical Artificial Intelligence course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Artificial Intelligence course.
Artificial Intelligence Course

71. Attend The Business Analytics Course From ExcelR. Practical Business Analytics Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analytics Course.

Simple Linear Regression

Correlation vs Covariance

Correlation vs Covariance
Simple linear regression
data science interview questions

74. Cool stuff you have and you keep overhaul every one of us

data science interview questions

75. Very interesting blog Thank you for sharing such a nice and interesting blog and really very helpful article.

76. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites!

Simple Linear Regression

Correlation vs Covariance

Correlation vs Covariance
Simple linear regression
data science interview questions

78. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.

data analytics courses

79. Attend online training from one of the best training institute Data Science Training in Hyderabad

80. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.

Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.

Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.

The Nodejs Training Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

81. This Information Which You Shared Was Really Fantastic

82. Attend The Data Analyst Course From ExcelR. Practical Data Analyst Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analyst Course.
Data Analyst Course

83. Attend The Data Analyst Course From ExcelR. Practical Data Analyst Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analyst Course.
Data Analyst Course

84. Attend The Data Scientist Courses From ExcelR. Practical Data Scientist Courses Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Scientist Courses. Data Scientist Courses

85. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.

Simple Linear Regression

Correlation vs Covariance

86. It has fully emerged to crown Singapore's southern shores and undoubtedly placed her on the global map of residential landmarks. I still scored the more points than I ever have in a season for GS. I think you would be hard pressed to find somebody with the same consistency I have had over the years so I am happy with that.
machine learning courses in bangalore

87. Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!

Simple Linear Regression

Correlation vs covariance

KNN Algorithm

88. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science certification

89. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Simple Linear Regression
Correlation vs covariance
data science interview questions
KNN Algorithm

90. Very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing, data science online course

91. Thanks for such a great article here. I was searching for something like this for quite a long time and at last, I’ve found it on your blog. It was definitely interesting for me to read about their market situation nowadays.Also Checkoutdata science course in Hyderabad

92. Your article was so impressive and informative. Its very interesting to read. Thanks for sharing,data scientist courses

93. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Simple Linear Regression
Correlation vs covariance
data science interview questions
KNN Algorithm

94. Impressive blog with lovely information. really very useful article for us thanks for sharing such a wonderful blog. data science training in Hyderabad

95. Attend The Artificial Intelligence course From ExcelR. Practical Artificial Intelligence course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Artificial Intelligence course.
Artificial Intelligence Course

96. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.

Simple Linear Regression

Correlation vs Covariance

97. I would you like to say thank you so much for my heart. Really amazing and impressive post you have the share. Please keep sharing

98. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.

Simple Linear Regression

Correlation vs covariance

KNN Algorithm

Logistic Regression explained

99. I like your post. Everyone should do read this blog. Because this blog is important for all now I will share this post. Thank you so much for share with us.

100. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.

data science interview questions

101. Nice information thanks for sharing it’s very useful. This article gives me so much information.

102. I really loved reading your blog. I also found your posts very interesting. In fact, after reading, I had to go show it to my friend and he enjoyed it as well!!!!

103. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Simple Linear Regression
Correlation vs covariance
data science interview questions
KNN Algorithm
Logistic Regression explained

104. Highly informative article. This site has lots of information and it is useful for us. Thanks for sharing your views.

105. "I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it. data science courses
"

106. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance
Simple Linear Regression
data science interview questions
KNN Algorithm
Logistic Regression explained

107. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course in hyderabad with placements

108. Actually I read it yesterday but I had some thoughts about it and today I wanted to read it again because it is very well written.
artificial intelligence course in bangalore