Internet blockchain big data
1. blockchain: it is a new application mode of distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and other computer technologies. The so-called consensus mechanism is a mathematical algorithm to establish trust and obtain interests between different nodes in the blockchain system
blockchain is an important concept of bitcoin. According to the 2014-2016 global bitcoin Development Research Report issued by the Internet Finance Laboratory of Wukou School of finance, Tsinghua University and Sina Technology, blockchain is the underlying technology and infrastructure of bitcoin [2]. In essence, it is a decentralized database, as well as the underlying technology of bitcoin. Blockchain is a series of data blocks generated by cryptography. Each data block contains the information of a bitcoin network transaction, which is used to verify the validity of the information (anti-counterfeiting) and generate the next block
2. Big data: refers to the data set that cannot be captured, managed and processed by conventional software tools within a certain period of time. It is a massive, high growth rate and diversified information asset that needs new processing mode to have stronger decision-making power, insight and discovery power and process optimization ability
in the near future, the number of devices in the Internet of things will explode, which may be hundreds of billions or trillions. If such a huge network is still managed in a centralized networking mode, the infrastructure investment and maintenance of the data center should be incalculable
in essence, big data belongs to a small branch of database, so this problem is attributed to the relationship between big data and database. Database is a very old research field in software, Internet and it. From the initial file system to ER model, to the three achievements of traditional database, which are known to all, relational model, transaction processing and query optimization, to the rise of NoSQL database after the popularity of Internet, database technology is constantly developing and changing, Then it also includes semi-structured data processing represented by XML and unstructured data processing such as text and voice
it seems that the relationship between blockchain and database is actually such a relationship. From the evolution process of database technology, we can find that it always comes from how to meet new business requirements and then create new data processing technologies. For example, from the beginning of the file system, why do we need the ER model? It is because with the development of the financial instry, people have further requirements for these fast accounting, high concurrency data writing and access, which leads to the emergence and rapid development of the entity relationship model. Why did NoSQL database appear later? It is because the rapid development of the Internet puts forward higher requirements for updating the database, so in essence we think that the whole Internet is a big database
things are always developing. Of course, after we solve the problem of massive real-time data processing on the Internet through NoSQL database and cloud storage, the next problem must come, that is, how to solve the authenticity and effectiveness of data in a large-scale way
for example, it may be related to our diet, from the problem of food and clothing at the beginning, to the problem of nutrition structure, and then to the problem of food safety that everyone is concerned about. The development of database is the same. When we can solve these problems of data storage and data access through ER entity relationship model and NoSQL database, Next, we need to care about and solve the problem of authenticity and effectiveness
so at this stage, these technologies, represented by blockchain, must be a new starting point and new requirements for the authenticity, validity, unforgeability and tamper resistance of data compared with the current database. We can clearly feel that the trend of database and blockchain integration is actually very close and irresistible, just like the movie just mentioned, the procers of content begin to develop in the direction of virtual reality and augmented reality; From the perspective of database, blockchain is a new way of data organization. We believe that big data and blockchain are integrated.
because of its immeasurable value, it has become an important research topic
big data is so valuable, but it can only be concentrated in the hands of a small number of owners. And people's life will proce a lot of data. The problem is that indivials have no way to obtain and store their own data e to external restrictions
the blockchain enables each indivial to have all the data generated by himself, and has a way to trade, analyze... And ensure the security of personal data.
in a word, blockchain is decentralized, using the trust protocol between distributed data nodes to achieve data sharing and mining upgrade. Artificial intelligence covers a wide range, including language recognition, image recognition, machine learning, intelligent logic and many other disciplines.
compared with pure big data technology, blockchain technology breaks the traditional big data monopoly, avoids the illegal factors of big data application, extends data application to all walks of life, makes data flow, complete storage, system analysis at ease, and makes use of its encryption and non tamperability, It not only ensures the authenticity and traceability of big data, but also solves the ownership problem of big data.
first, improve data quality
the essence of blockchain is a decentralized distributed ledger. It can also be understood as a non tampering, full history, distributed database storage technology. Therefore, blockchain technology can liberate more data. The trustworthiness, security and non tampering of blockchain technology fundamentally improve the quality of data and enhance the ability of data inspection
Second, dealing with the problem of data island
big data has a very serious problem of data island, and a lot of data cannot be obtained at present. Blockchain is expected to deal with this problem. This is mainly because blockchain is not only a distributed ledger, but also has the characteristics of decentralization and openness. As order maintainers in the financial market, regulatory organizations can also predict and analyze possible dangerous problems through the data chain in the blockchain
3. Dealing with data leakage
in essence, the blockchain is actually a decentralized database. Therefore, if the data of one node in the blockchain changes, other nodes will find it at the first time, so the possibility of data leakage will be greatly reced. Only in the form of private key can the identity information of each node in the blockchain be obtained successfully, and only the data owner can know the private key
4. Blockchain can protect the relevant rights and interests of data
for valuable data assets of indivials or organizations, blockchain can be used to register them, and transaction records are recognized, transparent and traceable throughout the network. The source, ownership, use right and circulation path of big data assets are clear, which is of great value to the transaction of data assets
v. traceability of blockchain
every step of data collection, trading, circulation, and accounting analysis can be kept on the blockchain, so that the data quality can obtain unprecedented strong trust endorsement. At the same time, it also ensures the correctness of data analysis results and the effect of data mining
sinomeni will share with you the role of blockchain in big data. If you are interested in big data engineering, I hope this article can help you. If you want to know more about the skills and materials of data analysts and big data engineers, you can click other articles on this website to learn
in terms of the connection, big data can be combined with blockchain technology. For example, blockchain has the characteristics of traceability. When big data packets are traded on the blockchain network, they can see the flow direction in the whole network. Be able to confirm and track the data packet