Influence of image quality on neural network computing power
Publish: 2021-05-13 04:22:45
1. The higher the influence pixel, the more complex the network structure is, the better the optimization technology is, the longer the training time is, and the setting of super parameters is
just like cifar data set and Imagenet data set face different data sets, the above components should change correspondingly
if the GPU is too small, image downsampling and batch can be considered_ The size should be smaller and the network structure should be compressed properly
just like cifar data set and Imagenet data set face different data sets, the above components should change correspondingly
if the GPU is too small, image downsampling and batch can be considered_ The size should be smaller and the network structure should be compressed properly
2. Let me give you a simple example. For example, we now build a network model to identify apples and oranges:
we need two sets of data, one is the eigenvalue, which is the input (P) of the network, the other is the tutor signal, Tell the network whether it is an orange or an apple (network output T):
our sample assumes this way (sampledata1. Txt):
P T
1031
2142
these two sets of data are interpreted in this way:
we assume that a water fruit is an orange or an apple by three features: shape, Color, taste, the first group of shape, color, taste are: 103 (of course, these numbers are randomly made up by me, this can be defined according to the actual situation). The fruit with the above characteristics is apple (t = 1), and the shape, color, taste are: 214, which shows that this is an orange (t = 2)
well, our network model is almost out. The input layer has three nodes (shape, color and taste), the output layer has one node (1 is apple, 2 is orange), and the hidden layer has one layer. We don't care about the number of nodes, because it's an empirical value. There are other parameter values that can be set in MATLAB, such as training function and training times, Now we start to train the network. First, we need to initialize the weights, input the first group of inputs: 103, and the network will output a value. Let's assume that it is 4. Then we calculate the error 4-1 = 3 according to the tutor signal (the correct tutor signal is 1, which means it is an apple). The error is transmitted to the BP neural network, and the neural network adjusts the weights according to the error, and then enters the second cycle, So let's input a set of data again: 204 (when you can still input 103, and if you input Apple's features all the time, the network will only recognize apples but not oranges. Now you understand your problem). Similarly, output a value and feed it back to the network again. This is the basic process of neural network training, Of course, these two sets of data are certainly not enough. If there are enough data, we will adjust the weights of the neural network to a very ideal state. What is the state? That is, after the network outputs again, the error is very small and less than the error value we require
the next step is the simulation and prediction_ 1 = sim (net, P), net is the network you set up, P is the input data, because the network weight has been determined, we do not need to know the value of t at this time, that is to say, we do not need to know whether it is an apple or an orange, and t_ 1 is the network forecast data, it may be 1 or 2, it may be 1.3, 2.2 and so on (most of them are such numbers), then you see this dozens close to 1 or 2, if it is 1.5, we think it is a hybrid of apple and orange, ha ha, laughing, meet X & lt= 0,5、x=1.5、x>= 2.5, I generally give up, which means unknown
in a word, you need to look for this information system. Since I am also engaged in image processing, I will give you a key reminder. If you use neural network to do image processing, you must have a good sample space, that is, your database must be standard. As for the mechanism of the network, training methods and so on, find an example to use matlab simulation to see the effect, and think about it by yourself. It is mainly about the setting of your hidden layer, the selection of training function and its convergence speed and error accuracy. It is impossible to introce you clearly in such a small space. The key is the sample, The extracted image features must have correlation, so that each threshold set is effective. OK, study hard, go to Matlab Chinese forum to find information, if not, go to Bau library, you don't need to use it all, why read a book! I wish you a smooth graation!
we need two sets of data, one is the eigenvalue, which is the input (P) of the network, the other is the tutor signal, Tell the network whether it is an orange or an apple (network output T):
our sample assumes this way (sampledata1. Txt):
P T
1031
2142
these two sets of data are interpreted in this way:
we assume that a water fruit is an orange or an apple by three features: shape, Color, taste, the first group of shape, color, taste are: 103 (of course, these numbers are randomly made up by me, this can be defined according to the actual situation). The fruit with the above characteristics is apple (t = 1), and the shape, color, taste are: 214, which shows that this is an orange (t = 2)
well, our network model is almost out. The input layer has three nodes (shape, color and taste), the output layer has one node (1 is apple, 2 is orange), and the hidden layer has one layer. We don't care about the number of nodes, because it's an empirical value. There are other parameter values that can be set in MATLAB, such as training function and training times, Now we start to train the network. First, we need to initialize the weights, input the first group of inputs: 103, and the network will output a value. Let's assume that it is 4. Then we calculate the error 4-1 = 3 according to the tutor signal (the correct tutor signal is 1, which means it is an apple). The error is transmitted to the BP neural network, and the neural network adjusts the weights according to the error, and then enters the second cycle, So let's input a set of data again: 204 (when you can still input 103, and if you input Apple's features all the time, the network will only recognize apples but not oranges. Now you understand your problem). Similarly, output a value and feed it back to the network again. This is the basic process of neural network training, Of course, these two sets of data are certainly not enough. If there are enough data, we will adjust the weights of the neural network to a very ideal state. What is the state? That is, after the network outputs again, the error is very small and less than the error value we require
the next step is the simulation and prediction_ 1 = sim (net, P), net is the network you set up, P is the input data, because the network weight has been determined, we do not need to know the value of t at this time, that is to say, we do not need to know whether it is an apple or an orange, and t_ 1 is the network forecast data, it may be 1 or 2, it may be 1.3, 2.2 and so on (most of them are such numbers), then you see this dozens close to 1 or 2, if it is 1.5, we think it is a hybrid of apple and orange, ha ha, laughing, meet X & lt= 0,5、x=1.5、x>= 2.5, I generally give up, which means unknown
in a word, you need to look for this information system. Since I am also engaged in image processing, I will give you a key reminder. If you use neural network to do image processing, you must have a good sample space, that is, your database must be standard. As for the mechanism of the network, training methods and so on, find an example to use matlab simulation to see the effect, and think about it by yourself. It is mainly about the setting of your hidden layer, the selection of training function and its convergence speed and error accuracy. It is impossible to introce you clearly in such a small space. The key is the sample, The extracted image features must have correlation, so that each threshold set is effective. OK, study hard, go to Matlab Chinese forum to find information, if not, go to Bau library, you don't need to use it all, why read a book! I wish you a smooth graation!
3. Convolutional neural network (CNN) is a kind of feed-forward neural network. Its artificial neurons can respond to some surrounding cells in the coverage area, and it has excellent performance for large-scale image processing[ 1] It includes an alternating convolutional layer and a pooling layer
convolutional neural network is an efficient recognition method which has been developed in recent years. In 1960s, when Hubel and Wiesel studied the neurons used for local sensitivity and direction selection in cat cerebral cortex, they found that their unique network structure could effectively rece the complexity of feedback neural network, and then proposed convolutional neural networks (CNN). Now, CNN has become one of the research hotspots in many scientific fields, especially in the field of pattern classification. Because the network avoids the complex pre-processing of the image and can directly input the original image, it has been more widely used. K. The new recognition machine proposed by Fukushima in 1980 is the first realization network of convolutional neural network. Later, more researchers improved the network. Among them, the representative research achievement is "improved cognitive machine" proposed by Alexander and Taylor, which combines the advantages of various improved methods and avoids time-consuming error back propagation.
convolutional neural network is an efficient recognition method which has been developed in recent years. In 1960s, when Hubel and Wiesel studied the neurons used for local sensitivity and direction selection in cat cerebral cortex, they found that their unique network structure could effectively rece the complexity of feedback neural network, and then proposed convolutional neural networks (CNN). Now, CNN has become one of the research hotspots in many scientific fields, especially in the field of pattern classification. Because the network avoids the complex pre-processing of the image and can directly input the original image, it has been more widely used. K. The new recognition machine proposed by Fukushima in 1980 is the first realization network of convolutional neural network. Later, more researchers improved the network. Among them, the representative research achievement is "improved cognitive machine" proposed by Alexander and Taylor, which combines the advantages of various improved methods and avoids time-consuming error back propagation.
4. Convolutional neural network has the following applications for research: 1. Shape recognition based on convolutional network is the basis of human visual system to analyze and recognize objects. Geometric shape is the expression of the essential characteristics of objects, and has the characteristics of translation, scaling and rotation invariance. Therefore, in the field of pattern recognition, it is of great significance for shape analysis and recognition, As a special case and component of 3D image, 2D image recognition is the basis of 3D image recognition. 2. Face detection based on convolution neural network convolution neural network is different from the traditional face detection method. It directly acts on the input samples, uses the samples to train the network, and finally realizes the detection task. It is a nonparametric face detection method, which can save a series of complex processes in traditional methods, such as modeling, parameter estimation, parameter test and model reconstruction. This paper focuses on the face of any size, position, posture, direction, skin color, facial expression and lighting conditions in the image. 3. In classical pattern recognition, character recognition system usually extracts features in advance. After extracting many features, we need to analyze the correlation of these features, find the most representative features, and remove the features that have nothing to do with classification and autocorrelation. However, the extraction of these features depends too much on people's experience and subjective consciousness. The difference of the extracted features has a great impact on the classification performance, and even the order of the extracted features will affect the final classification performance. At the same time, the quality of image preprocessing will also affect the feature extraction.
5. profound
6. Convolutional neural network has the following applications for research:
1. Shape recognition based on convolutional neural network
the shape of an object is the basis of human visual system to analyze and recognize objects. Geometric shape is the expression of the essential features of an object, and has the characteristics of translation, scaling and rotation invariance, It is very important for shape analysis and recognition. As a special case and component of three-dimensional image, two-dimensional image recognition is the basis of three-dimensional image recognition< 2. Face detection based on convolution neural network
convolution neural network is different from traditional face detection methods. It directly acts on the input samples, trains the network with the samples, and finally realizes the detection task. It is a nonparametric face detection method, which can save a series of complex processes in traditional methods, such as modeling, parameter estimation, parameter test and model reconstruction. This paper focuses on the face of any size, position, posture, direction, skin color, facial expression and lighting conditions in the image
3. Character recognition system
in classical pattern recognition, features are usually extracted in advance. After extracting many features, we need to analyze the correlation of these features, find the most representative features, and remove the features that have nothing to do with classification and autocorrelation. However, the extraction of these features depends too much on people's experience and subjective consciousness. The difference of the extracted features has a great impact on the classification performance, and even the order of the extracted features will affect the final classification performance. At the same time, the quality of image preprocessing will also affect the feature extraction.
1. Shape recognition based on convolutional neural network
the shape of an object is the basis of human visual system to analyze and recognize objects. Geometric shape is the expression of the essential features of an object, and has the characteristics of translation, scaling and rotation invariance, It is very important for shape analysis and recognition. As a special case and component of three-dimensional image, two-dimensional image recognition is the basis of three-dimensional image recognition< 2. Face detection based on convolution neural network
convolution neural network is different from traditional face detection methods. It directly acts on the input samples, trains the network with the samples, and finally realizes the detection task. It is a nonparametric face detection method, which can save a series of complex processes in traditional methods, such as modeling, parameter estimation, parameter test and model reconstruction. This paper focuses on the face of any size, position, posture, direction, skin color, facial expression and lighting conditions in the image
3. Character recognition system
in classical pattern recognition, features are usually extracted in advance. After extracting many features, we need to analyze the correlation of these features, find the most representative features, and remove the features that have nothing to do with classification and autocorrelation. However, the extraction of these features depends too much on people's experience and subjective consciousness. The difference of the extracted features has a great impact on the classification performance, and even the order of the extracted features will affect the final classification performance. At the same time, the quality of image preprocessing will also affect the feature extraction.
7. Generalization ability refers to the ability of machine learning algorithm to adapt to fresh samples, a kind of ability to predict new input categories
find the rules behind the data through learning, and for the data outside the learning set with the same rules, this trained network can give the appropriate output, which is called generalization ability
generally speaking, the more complex the neural network is, the higher the complexity it bears, and the larger the complexity capacity of describing rules. Of course, the better, and of course, it is not absolute. But it can explain the problem of the capacity of a container, and the stronger the generalization ability of the neural network is
we need to know that structural complexity and sample complexity, sample quality, initial weights, learning time and other factors will affect the generalization ability of neural networks. In order to ensure the strong generalization ability of neural network, people have done a lot of research and obtained many generalization methods, including pruning algorithm, construction algorithm and evolutionary algorithm
the generalization ability of artificial neural network is mainly e to the ability to derive efficient feature sets from training sets through unsupervised pre learning. Once the complex problem is transformed into the form expressed by these features, it will become simple. Conceptually, this is a bit like doing an intelligent coordinate transformation for the training set
for example, if the training set is a lot of face images, then if the pre training is done well, the features such as nose, eyes, mouth and various basic face shapes can be derived. If the classification is based on these features rather than pixels, the result will be much better. Although the large-scale neural network has a lot of parameters, but because the classification is based on a small number of features, it will not proce over fitting
at the same time, aiming at the shortcomings of neural network, such as easy to fall into local extremum, difficult to determine structure and poor generalization ability, support vector regression (SVR), which can solve the problems of small sample, nonlinear and high dimension, is introced to predict the development indexes of oil and gas fields.
find the rules behind the data through learning, and for the data outside the learning set with the same rules, this trained network can give the appropriate output, which is called generalization ability
generally speaking, the more complex the neural network is, the higher the complexity it bears, and the larger the complexity capacity of describing rules. Of course, the better, and of course, it is not absolute. But it can explain the problem of the capacity of a container, and the stronger the generalization ability of the neural network is
we need to know that structural complexity and sample complexity, sample quality, initial weights, learning time and other factors will affect the generalization ability of neural networks. In order to ensure the strong generalization ability of neural network, people have done a lot of research and obtained many generalization methods, including pruning algorithm, construction algorithm and evolutionary algorithm
the generalization ability of artificial neural network is mainly e to the ability to derive efficient feature sets from training sets through unsupervised pre learning. Once the complex problem is transformed into the form expressed by these features, it will become simple. Conceptually, this is a bit like doing an intelligent coordinate transformation for the training set
for example, if the training set is a lot of face images, then if the pre training is done well, the features such as nose, eyes, mouth and various basic face shapes can be derived. If the classification is based on these features rather than pixels, the result will be much better. Although the large-scale neural network has a lot of parameters, but because the classification is based on a small number of features, it will not proce over fitting
at the same time, aiming at the shortcomings of neural network, such as easy to fall into local extremum, difficult to determine structure and poor generalization ability, support vector regression (SVR), which can solve the problems of small sample, nonlinear and high dimension, is introced to predict the development indexes of oil and gas fields.
8. This depends on whether there is a fully connected layer in your convolutional neural network, because unlike the fully connected layer, the parameters in the convolutional neural network are the weights of the convolution kernel, independent of the input size. If it is full convolution network, then the input image resolution requirements are not high, can be arbitrary size, but how much should have an impact. However, if there is a fully connected layer, it is necessary to ensure that the size of the input image is consistent, and it can be scaled by linear interpolation.
9. 1、 In 1946, the Hungarian American scientist von Neumann put forward the principle of storing program, treating the program itself as data. Since then, the development of computer has made great progress, but the design of information memory and processor in "von Neumann architecture" has been used up to now, and the information transmission channel connecting memory and processor is still realized by bus. With the massive growth of the amount of data processed, the limited data transmission rate of the bus is known as the "von Neumann bottleneck" - especially the rise of mobile Internet, social network, Internet of things, cloud computing, high-throughput sequencing and so on, which makes the "von Neumann bottleneck" increasingly prominent, and the limitation of the lack of self correction ability of computers has become an obstacle to development
structural defects also lead to functional limitations. For example, in terms of efficiency, the power consumption of computer operation is higher - although the amount of information processed by human brain is no less than that of computer, the power consumption is obviously much lower. Therefore, learning more layers of neural network, so that the computer can better simulate the function of human brain, has become a research hotspot since the late last century
in these studies, the core research is the essential structural difference between "von Neumann architecture" and "human brain architecture" -- compared with computers, the information storage and processing of human brain are realized through synapses, so there is no obvious boundary. It is the plasticity of billions of synapses in the human brain, the neural changes (variability, modifiability, etc.) caused by various factors and conditions after a certain period of time, that makes the memory and learning functions of the human brain realized
the brain has three characteristics that the computer does not have: low power consumption (the energy consumption of the human brain is only about 20 watts, while the supercomputer currently used to simulate the human brain consumes several megawatts of energy); Fault tolerance (breaking a transistor can destroy a microprocessor, but neurons in the brain are dying all the time); There is also no need to program it (the brain will learn and change while interacting with the outside world, rather than following the fixed path and branches of the preset algorithm.)
this description can be said to be the ultimate ideal of "electric" brain
note: the earliest computers were also realized by analog circuits, and then developed into digital CPUs with only 0 and 1
today's computers use the so-called von Neumann architecture, which transfers data back and forth in a linear computing sequence between a CPU and a memory chip. This approach works well when dealing with numbers and executing programs that are written precisely, but it doesn't work well when dealing with pictures or sounds and understanding their meaning
one thing is very telling: in 2012, Google showed that its artificial intelligence software could learn to recognize the cat in the video without being told what the cat was, and 16000 processors were used to complete this task
to continue to improve the performance of these processors, manufacturers have to equip them with more and faster transistors, silicon memory buffers and data paths, but the heat generated by all these components limits the operation speed of chips, especially in mobile devices with limited power. This may hinder the development of devices that can effectively process pictures, sounds and other sensory information, as well as their application in tasks such as face recognition, robots, or transportation equipment and shipping
the neuromorphological chip attempts to imitate the human brain in a large-scale parallel way to process information: billions of neurons and billions of synapses respond to sensory inputs such as visual and sound stimuli
in response to images and sounds, these neurons also change the way they connect with each other. We call this process learning. The neuromorphological chip incorporates the brain inspired "neural network" model, so it can do the same thing
Jeff Hawkins, a leading thinker of artificial intelligence, said that it is too inefficient to try to simulate the human brain with special software on traditional processors (what Google did in the cat experiment) as the basis of continuously improving intelligence
Hawkins created the palm pilot and later co founded numenta, which makes software that draws inspiration from the human brain“ You can't just build it in software, "he says of AI." you have to build it in silicon. "
in the existing computer computing, the execution of the program is line by line, while the neural network computer is different
the current AI programs basically write all kinds of knowledge into statements one by one, and then pour them into the system. When a problem is entered into an intelligent program, it will search its own database and select the best or most recent solution. In 2011, IBM's famous Watson intelligent computer was the strongest champion of human beings who used such technology to defeat in American TV puzzle programs
(neural network computer) with this kind of asynchronous signal transmission (named because there is no central clock that can synchronize it), the speed of data processing is faster than that of synchronous signal transmission, thinking that there is no time to waste waiting for the clock to send a signal. Asynchronous signaling also consumes less energy, which satisfies the first characteristic of Dr. Meyer's ideal computer. If one of the processors fails, the system will bypass it from another route, which satisfies the second characteristic of Dr. Meyer's ideal computer. Because it is not easy to program for asynchronous signal transmission, most computer engineers ignore it. But it's perfect as a way to mimic the brain. Power consumption:
in terms of hardware, in recent years, it is mainly through the simulation of large-scale neural networks, such as Google brain, Microsoft Adam and so on. But these networks need a large number of traditional computer clusters. For example, Google brain uses 1000 computers with 16 core processors. Although this architecture shows considerable power, it still consumes a lot of energy. IBM, on the other hand, imitates it on a chip. 4096 cores, 1 million "neurons" and 256 million "synapses" are integrated into a few centimeters in diameter (1 / 16 the size of the 2011 prototype), and the energy consumption is less than 70 MW
the IBM research team has made a demonstration using the neovision2 tower dataset that has done DARPA. It can recognize people, bicycles, buses, trucks and so on in real time in the normal speed of 30 frames per second video taken from Stanford University Hoover tower intersection, with an accuracy rate of 80%. In contrast, it takes 100 times slower for a laptop to complete the same task, but the energy consumption is 10000 times that of IBM chip< br />
Ref: A million spiking-neuron integrated circuit with a scalable communication network and interface. Paul A. Merolla et al. Science 345, 668 (2014); Doi: 10.1126/science.1254642
e to the need for a database with a lot of data for training and powerful computing power for prediction, some existing deep learning, such as Andrew NG's Google brain and Apple's Siri, need to be connected to the cloud server< Second, controversy:
although deep learning has been applied to cutting-edge scientific research and daily life, Google has actually been carried in the core search function. However, other well-known AI laboratories have different responses to deep learning technology. For example, Oren Etzioni, executive director of Allen artificial intelligence center, did not consider integrating deep learning into the current development of artificial intelligence system. The current research of the Institute is aimed at scientific knowledge at primary school level, hoping to develop intelligent programs that can easily cope with all kinds of examinations just by reading school textbooks. Oren Etzioni takes aircraft as an example. He says that the most successful aircraft design does not come from imitating the structure of birds, so the analogy of brain and nerve can not guarantee the realization of artificial intelligence. Therefore, they do not consider using deep learning technology to develop this system for the moment
but in the short term, the situation may not be so optimistic< First of all, chip programming is still a big problem. The programming of the chip should consider which neuron to connect and the degree of interaction between neurons. For example, in order to identify the car in the above video, the programmer must first make necessary settings for the simulation version of the chip, and then transfer it to the actual chip. This kind of chip needs to subvert the traditional programming ideas. Although IBM released a set of tools last year, it is still very difficult to program at present. The IBM team is working out a development library to make the process easier Of course, this may not be a problem if we look back at the history of programming languages from assembly in the past.)
secondly, in the view of some professionals, the capability of this chip has yet to be confirmed
moreover, real cognitive computing should be able to learn from experience, find connections, put forward hypotheses, memorize, and learn based on results. In IBM's demonstration, all training is performed on the offline von Neumann computer. However, most of the current machine learning is offline, because learning often needs to adjust the algorithm, and IBM's hardware does not have the flexibility to adjust, so it is not good at doing this thing< Artificial neural network (ANN) is a mathematical model or calculation model that imitates the structure and function of biological neural network
neural network is a kind of operation model, which is composed of a large number of nodes (or "neurons", or "units") and their connections. Each node represents a specific output function, which is called activation function. The connection between each two nodes represents a weighted value for the signal passing through the connection, which is called weight, which is equivalent to the memory of artificial neural network. The output of the network varies with the connection mode, weight value and excitation function of the network. The network itself is usually the approximation of some algorithm or function in nature, or the expression of a logical strategy. Ref: Wikipedia: artificial neural network
circuit principle
the secretion of neurotransmitter in turn is the response to action potential stimulation. However, after receiving one of these neurotransmitter signals, neurons will not continue to emit action potentials. Of course, they will graally accumulate to a limit value. After a neuron receives a certain number of signals and exceeds the limit - essentially a simulated process - it then emits an action potential and resets itself. Spike's artificial neurons do the same thing. Every time they are activated, they accumulate charges in the capacitor until they reach the limit value, and then the capacitor discharges. The specific circuit structure and analysis will be updated if possible
at present, there are several ways to realize hardware, such as power (IBM, qcomm), analog power, digital analog hybrid (academia), GPU
structural defects also lead to functional limitations. For example, in terms of efficiency, the power consumption of computer operation is higher - although the amount of information processed by human brain is no less than that of computer, the power consumption is obviously much lower. Therefore, learning more layers of neural network, so that the computer can better simulate the function of human brain, has become a research hotspot since the late last century
in these studies, the core research is the essential structural difference between "von Neumann architecture" and "human brain architecture" -- compared with computers, the information storage and processing of human brain are realized through synapses, so there is no obvious boundary. It is the plasticity of billions of synapses in the human brain, the neural changes (variability, modifiability, etc.) caused by various factors and conditions after a certain period of time, that makes the memory and learning functions of the human brain realized
the brain has three characteristics that the computer does not have: low power consumption (the energy consumption of the human brain is only about 20 watts, while the supercomputer currently used to simulate the human brain consumes several megawatts of energy); Fault tolerance (breaking a transistor can destroy a microprocessor, but neurons in the brain are dying all the time); There is also no need to program it (the brain will learn and change while interacting with the outside world, rather than following the fixed path and branches of the preset algorithm.)
this description can be said to be the ultimate ideal of "electric" brain
note: the earliest computers were also realized by analog circuits, and then developed into digital CPUs with only 0 and 1
today's computers use the so-called von Neumann architecture, which transfers data back and forth in a linear computing sequence between a CPU and a memory chip. This approach works well when dealing with numbers and executing programs that are written precisely, but it doesn't work well when dealing with pictures or sounds and understanding their meaning
one thing is very telling: in 2012, Google showed that its artificial intelligence software could learn to recognize the cat in the video without being told what the cat was, and 16000 processors were used to complete this task
to continue to improve the performance of these processors, manufacturers have to equip them with more and faster transistors, silicon memory buffers and data paths, but the heat generated by all these components limits the operation speed of chips, especially in mobile devices with limited power. This may hinder the development of devices that can effectively process pictures, sounds and other sensory information, as well as their application in tasks such as face recognition, robots, or transportation equipment and shipping
the neuromorphological chip attempts to imitate the human brain in a large-scale parallel way to process information: billions of neurons and billions of synapses respond to sensory inputs such as visual and sound stimuli
in response to images and sounds, these neurons also change the way they connect with each other. We call this process learning. The neuromorphological chip incorporates the brain inspired "neural network" model, so it can do the same thing
Jeff Hawkins, a leading thinker of artificial intelligence, said that it is too inefficient to try to simulate the human brain with special software on traditional processors (what Google did in the cat experiment) as the basis of continuously improving intelligence
Hawkins created the palm pilot and later co founded numenta, which makes software that draws inspiration from the human brain“ You can't just build it in software, "he says of AI." you have to build it in silicon. "
in the existing computer computing, the execution of the program is line by line, while the neural network computer is different
the current AI programs basically write all kinds of knowledge into statements one by one, and then pour them into the system. When a problem is entered into an intelligent program, it will search its own database and select the best or most recent solution. In 2011, IBM's famous Watson intelligent computer was the strongest champion of human beings who used such technology to defeat in American TV puzzle programs
(neural network computer) with this kind of asynchronous signal transmission (named because there is no central clock that can synchronize it), the speed of data processing is faster than that of synchronous signal transmission, thinking that there is no time to waste waiting for the clock to send a signal. Asynchronous signaling also consumes less energy, which satisfies the first characteristic of Dr. Meyer's ideal computer. If one of the processors fails, the system will bypass it from another route, which satisfies the second characteristic of Dr. Meyer's ideal computer. Because it is not easy to program for asynchronous signal transmission, most computer engineers ignore it. But it's perfect as a way to mimic the brain. Power consumption:
in terms of hardware, in recent years, it is mainly through the simulation of large-scale neural networks, such as Google brain, Microsoft Adam and so on. But these networks need a large number of traditional computer clusters. For example, Google brain uses 1000 computers with 16 core processors. Although this architecture shows considerable power, it still consumes a lot of energy. IBM, on the other hand, imitates it on a chip. 4096 cores, 1 million "neurons" and 256 million "synapses" are integrated into a few centimeters in diameter (1 / 16 the size of the 2011 prototype), and the energy consumption is less than 70 MW
the IBM research team has made a demonstration using the neovision2 tower dataset that has done DARPA. It can recognize people, bicycles, buses, trucks and so on in real time in the normal speed of 30 frames per second video taken from Stanford University Hoover tower intersection, with an accuracy rate of 80%. In contrast, it takes 100 times slower for a laptop to complete the same task, but the energy consumption is 10000 times that of IBM chip< br />
Ref: A million spiking-neuron integrated circuit with a scalable communication network and interface. Paul A. Merolla et al. Science 345, 668 (2014); Doi: 10.1126/science.1254642
e to the need for a database with a lot of data for training and powerful computing power for prediction, some existing deep learning, such as Andrew NG's Google brain and Apple's Siri, need to be connected to the cloud server< Second, controversy:
although deep learning has been applied to cutting-edge scientific research and daily life, Google has actually been carried in the core search function. However, other well-known AI laboratories have different responses to deep learning technology. For example, Oren Etzioni, executive director of Allen artificial intelligence center, did not consider integrating deep learning into the current development of artificial intelligence system. The current research of the Institute is aimed at scientific knowledge at primary school level, hoping to develop intelligent programs that can easily cope with all kinds of examinations just by reading school textbooks. Oren Etzioni takes aircraft as an example. He says that the most successful aircraft design does not come from imitating the structure of birds, so the analogy of brain and nerve can not guarantee the realization of artificial intelligence. Therefore, they do not consider using deep learning technology to develop this system for the moment
but in the short term, the situation may not be so optimistic< First of all, chip programming is still a big problem. The programming of the chip should consider which neuron to connect and the degree of interaction between neurons. For example, in order to identify the car in the above video, the programmer must first make necessary settings for the simulation version of the chip, and then transfer it to the actual chip. This kind of chip needs to subvert the traditional programming ideas. Although IBM released a set of tools last year, it is still very difficult to program at present. The IBM team is working out a development library to make the process easier Of course, this may not be a problem if we look back at the history of programming languages from assembly in the past.)
secondly, in the view of some professionals, the capability of this chip has yet to be confirmed
moreover, real cognitive computing should be able to learn from experience, find connections, put forward hypotheses, memorize, and learn based on results. In IBM's demonstration, all training is performed on the offline von Neumann computer. However, most of the current machine learning is offline, because learning often needs to adjust the algorithm, and IBM's hardware does not have the flexibility to adjust, so it is not good at doing this thing< Artificial neural network (ANN) is a mathematical model or calculation model that imitates the structure and function of biological neural network
neural network is a kind of operation model, which is composed of a large number of nodes (or "neurons", or "units") and their connections. Each node represents a specific output function, which is called activation function. The connection between each two nodes represents a weighted value for the signal passing through the connection, which is called weight, which is equivalent to the memory of artificial neural network. The output of the network varies with the connection mode, weight value and excitation function of the network. The network itself is usually the approximation of some algorithm or function in nature, or the expression of a logical strategy. Ref: Wikipedia: artificial neural network
circuit principle
the secretion of neurotransmitter in turn is the response to action potential stimulation. However, after receiving one of these neurotransmitter signals, neurons will not continue to emit action potentials. Of course, they will graally accumulate to a limit value. After a neuron receives a certain number of signals and exceeds the limit - essentially a simulated process - it then emits an action potential and resets itself. Spike's artificial neurons do the same thing. Every time they are activated, they accumulate charges in the capacitor until they reach the limit value, and then the capacitor discharges. The specific circuit structure and analysis will be updated if possible
at present, there are several ways to realize hardware, such as power (IBM, qcomm), analog power, digital analog hybrid (academia), GPU
10. The theory of pooling is that the adjacent pixels in the image are related. Sampling every other line of an image, the results can still be seen. After a layer of convolution, the input image size changes little, but the convolution kernel-1 is reced. According to the correlation of adjacent data, in each NxN region, generally 2x2, a number represents the original four numbers
Hot content