Network Infrastructure Technology That Can Enhance IPFS Retrieval Speed
In recent years in the field of decentralized storage and data retrieval, the data retrieval speed of the Inter Planetary File System (IPFS) has an issue for a long time). However, a paper published in SIGCOMM, an international communications network summit, which is a technical breakthrough for blockchain storage industry for many years.
Enhancing data retrieval speed in IPFS by using ICN and NDN networks
The Special Interest Group on Data Communication (SIGCOMM) held its annual summit last year in Macau China. The summit was organized by the premier Association for Computing (ACM). At the two-day summit, (24th-26th September 2019), a paper was presented that changed everything.
This is significant because any paper presented in that summit could very well become tomorrow’s’ technology.
A team of eight scientists presented a solution that will change the IPFS ecosystem for the best. This solution presented in crafted solutions that can be integrated into IPFS. The scientists were from the University of London, the University of California, Los Angeles, and Osaka University in Japan.
Their solution entails the use of IP (internet protocol) over information-centric networking (ICN) (IP vs. ICN) for data retrieval in IPFS systems.
In their paper, the scientists compared the different performances of networking technologies. IP was compared alongside Named Data Networking (NDN)/ICN in terms of speed of data delivery.
The paper noted the various difficulties that IPFS currently faces. The paper also noted that ICN/NDN improved the speeds of IPFS 20 fold or more.
This is important because content has become more interactive. Video content has been extremely difficult to deliver over TCP/IP systems. Transmission Control Protocol/Internet Protocol models haven’t worked well as far as rich data transfer is concerned.
The future direction of technology: decentralized storage and data retrieval
Over the past five years, global IP traffic has increased nearly three times, and the core driving force behind the increase in traffic is rapid demand access for video content. According to authoritative data, the amount of video-on-demand data will double again by 2021. Currently, many content producers rely heavily on CND vendors to distribute their content to more end users. Although the more popular content producers are willing to pay more traffic distribution fees, these content producers generally tend to reduce this part of the cost.
On the other hand, there is highly competitive in the CDN market. The network and hardware costs for large CDN companies are very high. At the same time, CDN services profits are relatively low, and the source of profits mainly depends on additional services. Under the circumstances, peer-to-peer (P2P) networks have incredible advantages.
In general, a peer-to-peer network can retrieve content through the storage devices of nearby nodes, thereby reduce loading for the entire network. What is more, the peer-to-peer network has self-enhancement capability. The content becomes more popular, the more nodes are caching content data, so the cost of storing data becomes very low. However, the current peer-to-peer network is mainly used for downloading large files, and it has not been popularized among user group in-general. To some extent, the peer-to-peer network has certain commercial potential.
Above the concept of P2P network, the industry has further developed the distributed storage concept. Distributed storage provides decentralized data storage and data retrieval capabilities. Distributed storage based on new protocols and mechanisms of the distributed ledger and cryptography, through the consensus mechanism with cryptographic cryptocurrency, to provide certain incentives to resource providers. Moreover, it can help market participants reach consensus on proofs of useful resources contributions in distributed storage, through the way of the verifiable market involving multiple parties, so as to benefit multiple parties.
With the continuous development of Internet business needs, the content of data is already the largest demand-side of Internet traffic today. Under such a background, the traditional TCP / IP architecture with the host addressing as the core architecture design, in today’s context is the main traffic with demand, which disadvantages are gradually exposed. Since 2010, the content-centric network (ICN) architecture design different from TCP / IP has gradually appeared in the field of scientific research and has received support from a large number of universities, enterprises, and governments. In recent years, open-source scientific research projects represented by NDN and CDN are among these representatives.
The IPFS Interplanetary File System (IPFS) is a content-centric design similar to ICN and NDN networks. The system maps the CID of the content to the host through a self-verification content hash of the content identifier, and human-readable names to be mapped. In addition, IPFS provides data block transmission services through the BitSwap protocol and rewards resource providers through FileCoin.
In the paper, scientists designed a test to verify the performance difference between the pure IPFS system and IPFS system in NDN network in terms of video data transmission.
Finally, the conclusion of the paper is: the retrieval performance of the pure IPFS system is relatively low, mainly due to fetching address speed issue of the DHT technology used by IPFS, and the transmission efficiency of the BitSwap protocol is too low, and it will pass different nodes transmit duplicate file fragments at the same time, causing some loss.
The comparison data shows that NDN-based IPFS has a 20 times improvement in addressing performance compared to pure IPFS; Moreover, NDN-based IPFS has increased 20-times in throughput compared to pure IPFS.
In the paper, scientists designed a test to verify the performance difference between the pure IPFS system and IPFS system in NDN network in terms of video data transmission.
The above picture shows some parameter settings from the experiment. A video file with a length of 10 minutes with a resolution of 480P is used for the experiment. The paper uses NS-3 to run the IPFS product model. NDN uses the NDN SIM client end. The routing cache function is temporarily disabled. There are three forwarding strategies of NDN are ASF, NCC, and Best route strategy respectively.
Finally, the conclusion of the paper is: the retrieval performance of the pure IPFS system is relatively low, mainly due to fetching address speed issue of the DHT technology used by IPFS, and the transmission efficiency of the BitSwap protocol is too low, and it will pass different nodes transmit duplicate file fragments at the same time, causing some loss.
The comparison data shows that NDN-based IPFS has a 20 times improvement in addressing performance compared to pure IPFS; Moreover, NDN-based IPFS has a 20-times increase in throughput compared to pure IPFS.
Summary
In the current Internet, large-scale content delivery such as video on demand (VoD) requires a large investment in a large-scale distributed infrastructure. Content providers (CPs) often request third-party content delivery networks (CDNs) or build their own expensive content delivery infrastructure to cope with peak demand and maintain an adequate quality of service (QoS), while Internet service providers (ISP) needs to reconfigure its network.
According to the paper, it explains the industry has taken the direction for the first step to improving the IPFS system. This solution uses the user’s storage space as a CDN cache, and delivers content with sufficient quality (that is, similar to CDN) while retrieving content. as reward users’ resource usage in the market. As similar candidates for such systems, it considers P2P storage recently and delivery systems that use new mechanisms, such as rewarding useful work (such as storage), while ensuring fairness and accountability through proof of password.
During the speech in SIGCOMM, the author introduced the popular IPFS and studied its performance of delivering VoD content locally within ISP. Their research results show that running IPFS (running on top of IP) has performance limitations, and supplementing it through the ICN network layer can significantly improve the quality of delivery. Then, they proposed and compared several forwarding strategies for ICN, which can effectively complete routing requests and balance the load between peers with limited uplink resources.
We believe that ICN / NDN technology applies into IPFS, the field of decentralized storage is bound with a new technological revolution. Welcome to the faster-interconnected world. One that is driven by ICN/NDN-based IPFS!
For more information, you can read the paper here.
Website: www.ndn.link