README.md 5.42 KB
Newer Older
Yulin's avatar
Yulin committed
1
# cidranger
Steven Allen's avatar
Steven Allen committed
2

3
Fast IP to CIDR block(s) lookup using trie in Golang, inspired by [IPv4 route lookup linux](https://vincent.bernat.im/en/blog/2017-ipv4-route-lookup-linux).  Possible use cases include detecting if a IP address is from published cloud provider CIDR blocks (e.g. 52.95.110.1 is contained in published AWS Route53 CIDR 52.95.110.0/24), IP routing rules, etc.
Yulin's avatar
Yulin committed
4

Steven Allen's avatar
Steven Allen committed
5 6 7 8 9 10
Forked from https://github.com/yl2chen/cidranger due to upstream inactivity.

[![GoDoc Reference](https://img.shields.io/badge/godoc-reference-5272B4.svg?style=flat-square)](https://godoc.org/github.com/libp2p/go-cidranger)
[![Build Status](https://img.shields.io/travis/libp2p/go-cidranger.svg?branch=master&style=flat-square)](https://travis-ci.org/libp2p/go-cidranger)
[![Coverage Status](https://img.shields.io/coveralls/libp2p/go-cidranger.svg?branch=master&style=flat-square)](https://coveralls.io/github/libp2p/go-cidranger?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/libp2p/go-cidranger?&style=flat-square)](https://goreportcard.com/report/github.com/libp2p/go-cidranger)
Yulin's avatar
Yulin committed
11

神楽坂帕琪's avatar
神楽坂帕琪 committed
12
This is visualization of a trie storing CIDR blocks `128.0.0.0/2` `192.0.0.0/2` `200.0.0.0/5` without path compression, the 0/1 number on the path indicates the bit value of the IP address at specified bit position, hence the path from root node to a child node represents a CIDR block that contains all IP ranges of its children, and children's children.
13 14
<p align="left"><img src="http://i.imgur.com/vSKTEBb.png" width="600" /></p>

15
Visualization of trie storing same CIDR blocks with path compression, improving both lookup speed and memory footprint.
16 17 18
<p align="left"><img src="http://i.imgur.com/JtaDlD4.png" width="600" /></p>

## Getting Started
Yulin's avatar
Yulin committed
19
Configure imports.
Yulin Chen's avatar
Yulin Chen committed
20
```go
Yulin's avatar
Yulin committed
21
import (
Yulin Chen's avatar
Yulin Chen committed
22
  "net"
23

Steven Allen's avatar
Steven Allen committed
24
  "github.com/libp2p/go-cidranger"
Yulin's avatar
Yulin committed
25 26
)
```
27
Create a new ranger implemented using Path-Compressed prefix trie.
Yulin Chen's avatar
Yulin Chen committed
28
```go
29
ranger := NewPCTrieRanger()
Yulin's avatar
Yulin committed
30 31
```
Inserts CIDR blocks.
Yulin Chen's avatar
Yulin Chen committed
32
```go
Yulin's avatar
Yulin committed
33 34
_, network1, _ := net.ParseCIDR("192.168.1.0/24")
_, network2, _ := net.ParseCIDR("128.168.1.0/24")
35 36 37 38 39 40 41 42 43
ranger.Insert(NewBasicRangerEntry(*network1))
ranger.Insert(NewBasicRangerEntry(*network2))
```
To attach any additional value(s) to the entry, simply create custom struct
storing the desired value(s) that implements the RangerEntry interface:
```go
type RangerEntry interface {
	Network() net.IPNet
}
Yulin's avatar
Yulin committed
44 45
```
The prefix trie can be visualized as:
46
```
Yulin's avatar
Yulin committed
47 48 49 50 51
0.0.0.0/0 (target_pos:31:has_entry:false)
| 1--> 128.0.0.0/1 (target_pos:30:has_entry:false)
| | 0--> 128.168.1.0/24 (target_pos:7:has_entry:true)
| | 1--> 192.168.1.0/24 (target_pos:7:has_entry:true)
```
Yulin Chen's avatar
Yulin Chen committed
52 53
To test if given IP is contained in constructed ranger,
```go
Yulin's avatar
Yulin committed
54 55 56 57
contains, err = ranger.Contains(net.ParseIP("128.168.1.0")) // returns true, nil
contains, err = ranger.Contains(net.ParseIP("192.168.2.0")) // returns false, nil
```
To get all the networks given is contained in,
Yulin Chen's avatar
Yulin Chen committed
58
```go
Yulin's avatar
Yulin committed
59 60
containingNetworks, err = ranger.ContainingNetworks(net.ParseIP("128.168.1.0"))
```
61 62 63 64 65
To get all networks in ranger,
```go
entries, err := ranger.CoveredNetworks(*AllIPv4) // for IPv4
entries, err := ranger.CoveredNetworks(*AllIPv6) // for IPv6
```
Yulin's avatar
Yulin committed
66

67
## Benchmark
68
Compare hit/miss case for IPv4/IPv6 using PC trie vs brute force implementation, Ranger is initialized with published AWS ip ranges (889 IPv4 CIDR blocks and 360 IPv6)
Yulin Chen's avatar
Yulin Chen committed
69
```go
70 71 72
// Ipv4 lookup hit scenario
BenchmarkPCTrieHitIPv4UsingAWSRanges-4         	 5000000	       353   ns/op
BenchmarkBruteRangerHitIPv4UsingAWSRanges-4    	  100000	     13719   ns/op
Yulin's avatar
Yulin committed
73

74 75
// Ipv6 lookup hit scenario, counter-intuitively faster then IPv4 due to less IPv6 CIDR
// blocks in the AWS dataset, hence the constructed trie has less path splits and depth.
76 77
BenchmarkPCTrieHitIPv6UsingAWSRanges-4         	10000000	       143   ns/op
BenchmarkBruteRangerHitIPv6UsingAWSRanges-4    	  300000	      5178   ns/op
Yulin's avatar
Yulin committed
78

79 80 81
// Ipv4 lookup miss scenario
BenchmarkPCTrieMissIPv4UsingAWSRanges-4        	20000000	        96.5 ns/op
BenchmarkBruteRangerMissIPv4UsingAWSRanges-4   	   50000	     24781   ns/op
Yulin's avatar
Yulin committed
82

83 84 85
// Ipv6 lookup miss scenario
BenchmarkPCTrieHMissIPv6UsingAWSRanges-4       	10000000	       115   ns/op
BenchmarkBruteRangerMissIPv6UsingAWSRanges-4   	  100000	     10824   ns/op
Yulin's avatar
Yulin committed
86 87
```

88 89
## Example of IPv6 trie:
```
Yulin's avatar
Yulin committed
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109
::/0 (target_pos:127:has_entry:false)
| 0--> 2400::/14 (target_pos:113:has_entry:false)
| | 0--> 2400:6400::/22 (target_pos:105:has_entry:false)
| | | 0--> 2400:6500::/32 (target_pos:95:has_entry:false)
| | | | 0--> 2400:6500::/39 (target_pos:88:has_entry:false)
| | | | | 0--> 2400:6500:0:7000::/53 (target_pos:74:has_entry:false)
| | | | | | 0--> 2400:6500:0:7000::/54 (target_pos:73:has_entry:false)
| | | | | | | 0--> 2400:6500:0:7000::/55 (target_pos:72:has_entry:false)
| | | | | | | | 0--> 2400:6500:0:7000::/56 (target_pos:71:has_entry:true)
| | | | | | | | 1--> 2400:6500:0:7100::/56 (target_pos:71:has_entry:true)
| | | | | | | 1--> 2400:6500:0:7200::/56 (target_pos:71:has_entry:true)
| | | | | | 1--> 2400:6500:0:7400::/55 (target_pos:72:has_entry:false)
| | | | | | | 0--> 2400:6500:0:7400::/56 (target_pos:71:has_entry:true)
| | | | | | | 1--> 2400:6500:0:7500::/56 (target_pos:71:has_entry:true)
| | | | | 1--> 2400:6500:100:7000::/54 (target_pos:73:has_entry:false)
| | | | | | 0--> 2400:6500:100:7100::/56 (target_pos:71:has_entry:true)
| | | | | | 1--> 2400:6500:100:7200::/56 (target_pos:71:has_entry:true)
| | | | 1--> 2400:6500:ff00::/64 (target_pos:63:has_entry:true)
| | | 1--> 2400:6700:ff00::/64 (target_pos:63:has_entry:true)
| | 1--> 2403:b300:ff00::/64 (target_pos:63:has_entry:true)
Yulin's avatar
Yulin committed
110
```