Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to cilium/ebpf #29

Merged
merged 7 commits into from
Mar 31, 2020
Merged

Migrate to cilium/ebpf #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b4b1a7f
Remove CGO dependency by adopting cilium/ebpf
ti-mo Mar 27, 2020
63c1c99
magefile - build with CGO_ENABLED=0
ti-mo Mar 29, 2020
201329f
README.md - rephrase overview, add compatibility chart
ti-mo Mar 29, 2020
e73cf57
bpf - strongly-typed map definitions, use __inline instead of __attri…
ti-mo Mar 31, 2020
8128484
pkg/bpf - clean up trace and perf events in Close()
ti-mo Mar 31, 2020
cc1d4c8
pkg/bpf - remove overengineered errWorker infrastructure. unmarshal e…
ti-mo Mar 31, 2020
ebd6d61
bpf/acct.c - downsize initial flow_{cooldown,origin} max_entries
ti-mo Mar 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 23 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,35 @@

Conntracct is a tool for extracting network flow information from Linux hosts,
firewalls, gateways, container or virtualization hosts, even mid- to high-end
embedded devices. It does not capture or analyze packets in any way, but hooks
into Conntrack's accounting (acct) subsystem using eBPF to obtain packet/byte
counters, minimizing overhead.
embedded devices.

It does not capture or analyze packets in any way, but hooks into Conntrack's
accounting subsystem using eBPF with minimal overhead.

---

## Overview

Conntracct is built around a real-time metrics pipeline that supports one or
more time-series sinks to be attached. This allows for sending real-time
packet/byte counters to eg. InfluxDB, where they can be queried for a live
visualization.
Conntracct contains a metrics pipeline that supports shipping packet/byte
counters for individial network flows to backends like InfluxDB and
ElasticSearch, where they can be queried and visualized in real time.

## Compatibility

The following major distributions are supported:

The project aims to support unmodified kernels of most major distributions
that are Linux 4.9 or higher. No compatibility matrix yet, but some versions of
Fedora and Arch up to kernels 4.19 have been tested.
- Debian
- Stretch
- Buster
- Ubuntu
- Bionic
- Fedora
- Arch Linux

No flashy tech demo yet, Coming Soon. (tm)
The minimum required kernel version is 4.9. For distributions with rolling
releases, breakage is expected as the kernel's internal data structures evolve
over time. Please create an issue if you encounter any issues running the
project on rolling distributions.

## Roadmap

Expand All @@ -36,7 +47,7 @@ This is a small list of features that are planned to
- [ ] `conntracct test` subcommand to ship eBPF test suite with the binary
- [ ] ARMv7 (aarch64) support (Odroid XU3/4+, RPi 3+, etc.)
- [ ] Easy build procedure for targeting a single custom kernel
- [ ] Pure-go eBPF implementation without Cgo (https://github.com/newtools/ebpf)
- [x] Pure-go eBPF implementation without Cgo (https://github.com/cilium/ebpf)

## Installing

Expand Down
72 changes: 26 additions & 46 deletions bpf/acct.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,53 +45,47 @@ const int ready_val = 0x90;
// perf map to send update events to userspace.
struct bpf_map_def SEC("maps/perf_acct_update") perf_acct_update = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(__u32),
.max_entries = 1024,
};

// perf map to send destroy events to userspace.
struct bpf_map_def SEC("maps/perf_acct_end") perf_acct_end = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(__u32),
.max_entries = 1024,
};

// Hash that holds a kernel timestamp per flow indicating when
// the flow may send its next update event to userspace.
struct bpf_map_def SEC("maps/flow_cooldown") flow_cooldown = {
.type = BPF_MAP_TYPE_HASH,
.key_size = sizeof(int),
.value_size = sizeof(__u64),
.max_entries = 512000,
.key_size = sizeof(u64),
.value_size = sizeof(u64),
.max_entries = 65535,
};

// Hash that holds a timestamp per flow indicating when the flow
// was first seen. Used to implement age-based event rate limiting.
struct bpf_map_def SEC("maps/flow_origin") flow_origin = {
.type = BPF_MAP_TYPE_HASH,
.key_size = sizeof(int),
.value_size = sizeof(__u64),
.max_entries = 512000,
.key_size = sizeof(struct nf_conn *),
.value_size = sizeof(u64),
.max_entries = 65535,
};

// Communication channel between the kprobe and the kretprobe.
// Holds a pointer to the nf_conn in the hot path (kprobe) and
// reads + deletes it in the kretprobe.
struct bpf_map_def SEC("maps/currct") currct = {
.type = BPF_MAP_TYPE_PERCPU_HASH,
.key_size = sizeof(int),
.value_size = sizeof(void *),
.key_size = sizeof(u32),
.value_size = sizeof(struct nf_conn *),
.max_entries = 2048,
};

// Map holding configuration values for this BPF program.
// Indexed by enum o_config.
struct bpf_map_def SEC("maps/config") config = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(void *),
.key_size = sizeof(enum o_config),
.value_size = sizeof(u64),
.max_entries = ConfigMax,
};

Expand All @@ -100,15 +94,14 @@ struct bpf_map_def SEC("maps/config") config = {
// Indexed by enum o_config_ratecurve.
struct bpf_map_def SEC("maps/config_ratecurve") config_ratecurve = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(void *),
.key_size = sizeof(enum o_config_ratecurve),
.value_size = sizeof(u64),
.max_entries = ConfigCurveMax,
};

// probe_ready reads the `config` array map for the Ready flag.
// It returns true if the Ready flag is set to 0x90 (go).
__attribute__((always_inline))
static bool probe_ready() {
static __inline bool probe_ready() {

u64 oc_ready = ConfigReady;
u64 *rp = bpf_map_lookup_elem(&config, &oc_ready);
Expand All @@ -118,8 +111,7 @@ static bool probe_ready() {

// get_acct_ext gets a reference to the nf_conn's accounting extension.
// Returns non-zero on error.
__attribute__((always_inline))
static int get_acct_ext(struct nf_conn_acct **acct_ext, struct nf_conn *ct) {
static __inline int get_acct_ext(struct nf_conn_acct **acct_ext, struct nf_conn *ct) {

// Check if accounting extension is enabled and initialized
// for this connection. Important because the acct codepath
Expand All @@ -145,8 +137,7 @@ static int get_acct_ext(struct nf_conn_acct **acct_ext, struct nf_conn *ct) {

// get_ts_ext gets a reference to the nf_conn's timestamp extension.
// Returns non-zero on error.
__attribute__((always_inline))
static int get_ts_ext(struct nf_conn_tstamp **ts_ext, struct nf_conn *ct) {
static __inline int get_ts_ext(struct nf_conn_tstamp **ts_ext, struct nf_conn *ct) {

struct nf_ct_ext *ct_ext;
bpf_probe_read(&ct_ext, sizeof(ct_ext), &ct->ext);
Expand All @@ -167,8 +158,7 @@ static int get_ts_ext(struct nf_conn_tstamp **ts_ext, struct nf_conn *ct) {

// extract_counters extracts accounting info from an nf_conn into acct_event_t.
// Returns 0 if acct extension was present in ct.
__attribute__((always_inline))
static int extract_counters(struct acct_event_t *data, struct nf_conn *ct) {
static __inline int extract_counters(struct acct_event_t *data, struct nf_conn *ct) {

struct nf_conn_acct *acct_ext = 0;
if (get_acct_ext(&acct_ext, ct))
Expand All @@ -188,8 +178,7 @@ static int extract_counters(struct acct_event_t *data, struct nf_conn *ct) {

// extract_tstamp extracts the start timestamp of nf_conn_tstamp inside an nf_conn
// into acct_event_t. Returns 0 if timestamp extension was present in ct.
__attribute__((always_inline))
static int extract_tstamp(struct acct_event_t *data, struct nf_conn *ct) {
static __inline int extract_tstamp(struct acct_event_t *data, struct nf_conn *ct) {

struct nf_conn_tstamp *ts_ext = 0;
if (get_ts_ext(&ts_ext, ct))
Expand All @@ -202,8 +191,7 @@ static int extract_tstamp(struct acct_event_t *data, struct nf_conn *ct) {

// extract_tuple extracts tuple information (proto, src/dest ip and port) of an nf_conn
// into an acct_event_t.
__attribute__((always_inline))
static void extract_tuple(struct acct_event_t *data, struct nf_conn *ct) {
static __inline void extract_tuple(struct acct_event_t *data, struct nf_conn *ct) {

struct nf_conntrack_tuple_hash tuplehash[IP_CT_DIR_MAX];
bpf_probe_read(&tuplehash, sizeof(tuplehash), &ct->tuplehash);
Expand All @@ -219,8 +207,7 @@ static void extract_tuple(struct acct_event_t *data, struct nf_conn *ct) {
}

// extract_netns extracts the nf_conn's network namespace inode number into an acct_event_t.
__attribute__((always_inline))
static void extract_netns(struct acct_event_t *data, struct nf_conn *ct) {
static __inline void extract_netns(struct acct_event_t *data, struct nf_conn *ct) {

// Obtain reference to network namespace.
// Warning: ct_net is a possible_net_t with a single member,
Expand All @@ -237,8 +224,7 @@ static void extract_netns(struct acct_event_t *data, struct nf_conn *ct) {

// curve_get returns an entry from the curve array as a signed 64-bit integer.
// Returns negative if an entry was not found at the requested index.
__attribute__((always_inline))
static s64 curve_get(enum o_config_ratecurve curve_enum) {
static __inline s64 curve_get(enum o_config_ratecurve curve_enum) {

int offset = curve_enum;
u64 *confp = bpf_map_lookup_elem(&config_ratecurve, &offset);
Expand All @@ -249,8 +235,7 @@ static s64 curve_get(enum o_config_ratecurve curve_enum) {
}

// flow_cooldown_expired returns true if the flow's cooldown period is over.
__attribute__((always_inline))
static bool flow_cooldown_expired(struct nf_conn *ct, u64 ts) {
static __inline bool flow_cooldown_expired(struct nf_conn *ct, u64 ts) {

// Look up the flow's cooldown expiration time.
u64 *nextp = bpf_map_lookup_elem(&flow_cooldown, &ct);
Expand All @@ -268,8 +253,7 @@ static bool flow_cooldown_expired(struct nf_conn *ct, u64 ts) {
// the second age threshold (curve1age), to protect against event storms
// when the program is restarted.
// This call is write-once due to BPF_NOEXIST.
__attribute__((always_inline))
static u64 flow_initialize_origin(struct nf_conn *ct, u64 ts, u64 pkts_total) {
static __inline u64 flow_initialize_origin(struct nf_conn *ct, u64 ts, u64 pkts_total) {

u64 origin = ts;

Expand Down Expand Up @@ -299,8 +283,7 @@ static u64 flow_initialize_origin(struct nf_conn *ct, u64 ts, u64 pkts_total) {
// hashmap. The time elapsed between the origin and the given
// ts is returned. If there is no first-seen timestamp for the
// flow, returns a zero value.
__attribute__((always_inline))
static u64 flow_get_age(struct nf_conn *ct, u64 ts) {
static __inline u64 flow_get_age(struct nf_conn *ct, u64 ts) {

// Initialize origin to the current timestamp so a lookup miss
// causes a 0ns age to be returned. (new or unknown flows)
Expand All @@ -317,8 +300,7 @@ static u64 flow_get_age(struct nf_conn *ct, u64 ts) {
// for the flow during the current event.
// Returns negative if the flow is younger than the minimum age threshold,
// or if an internal curve lookup error occurred.
__attribute__((always_inline))
static s64 flow_get_interval(struct nf_conn *ct, u64 ts) {
static __inline s64 flow_get_interval(struct nf_conn *ct, u64 ts) {

// Always returns a positive or 0 value.
u64 age = flow_get_age(ct, ts);
Expand Down Expand Up @@ -346,8 +328,7 @@ static s64 flow_get_interval(struct nf_conn *ct, u64 ts) {
return curve_get(ConfigCurve2Interval);
}

__attribute__((always_inline))
static u64 flow_set_cooldown(struct nf_conn *ct, u64 ts) {
static __inline u64 flow_set_cooldown(struct nf_conn *ct, u64 ts) {

// Get the update interval for this flow.
// A negative result indicates that the event should be dropped
Expand All @@ -365,8 +346,7 @@ static u64 flow_set_cooldown(struct nf_conn *ct, u64 ts) {
}

// flow_cleanup removes all possible map entries related to the connection.
__attribute__((always_inline))
static void flow_cleanup(struct nf_conn *ct) {
static __inline void flow_cleanup(struct nf_conn *ct) {
bpf_map_delete_elem(&flow_cooldown, &ct);
bpf_map_delete_elem(&flow_origin, &ct);
}
Expand Down
41 changes: 12 additions & 29 deletions bpf/bpf_helpers.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,18 +107,18 @@ static int (*bpf_l4_csum_replace)(void *ctx, int off, int from, int to, int flag
#define PT_REGS_SP(x) ((x)->sp)
#define PT_REGS_IP(x) ((x)->ip)

#elif defined(__s390x__)

#define PT_REGS_PARM1(x) ((x)->gprs[2])
#define PT_REGS_PARM2(x) ((x)->gprs[3])
#define PT_REGS_PARM3(x) ((x)->gprs[4])
#define PT_REGS_PARM4(x) ((x)->gprs[5])
#define PT_REGS_PARM5(x) ((x)->gprs[6])
#define PT_REGS_RET(x) ((x)->gprs[14])
#define PT_REGS_FP(x) ((x)->gprs[11]) /* Works only with CONFIG_FRAME_POINTER */
#define PT_REGS_RC(x) ((x)->gprs[2])
#define PT_REGS_SP(x) ((x)->gprs[15])
#define PT_REGS_IP(x) ((x)->ip)
#elif defined(__arm__)

#define PT_REGS_PARM1(x) ((x)->uregs[0])
#define PT_REGS_PARM2(x) ((x)->uregs[1])
#define PT_REGS_PARM3(x) ((x)->uregs[2])
#define PT_REGS_PARM4(x) ((x)->uregs[3])
#define PT_REGS_PARM5(x) ((x)->uregs[4])
#define PT_REGS_RET(x) ((x)->uregs[14])
#define PT_REGS_FP(x) ((x)->uregs[11]) /* Works only with CONFIG_FRAME_POINTER */
#define PT_REGS_RC(x) ((x)->uregs[0])
#define PT_REGS_SP(x) ((x)->uregs[13])
#define PT_REGS_IP(x) ((x)->uregs[12])

#elif defined(__aarch64__)

Expand All @@ -133,28 +133,11 @@ static int (*bpf_l4_csum_replace)(void *ctx, int off, int from, int to, int flag
#define PT_REGS_SP(x) ((x)->sp)
#define PT_REGS_IP(x) ((x)->pc)

#elif defined(__powerpc__)

#define PT_REGS_PARM1(x) ((x)->gpr[3])
#define PT_REGS_PARM2(x) ((x)->gpr[4])
#define PT_REGS_PARM3(x) ((x)->gpr[5])
#define PT_REGS_PARM4(x) ((x)->gpr[6])
#define PT_REGS_PARM5(x) ((x)->gpr[7])
#define PT_REGS_RC(x) ((x)->gpr[3])
#define PT_REGS_SP(x) ((x)->sp)
#define PT_REGS_IP(x) ((x)->nip)

#endif

#ifdef __powerpc__
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ (ip) = (ctx)->link; })
#define BPF_KRETPROBE_READ_RET_IP BPF_KPROBE_READ_RET_IP
#else
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ \
bpf_probe_read(&(ip), sizeof(ip), (void *)PT_REGS_RET(ctx)); })
#define BPF_KRETPROBE_READ_RET_IP(ip, ctx) ({ \
bpf_probe_read(&(ip), sizeof(ip), \
(void *)(PT_REGS_FP(ctx) + sizeof(ip))); })
#endif

#endif
7 changes: 4 additions & 3 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@ module github.com/ti-mo/conntracct

go 1.12

replace github.com/cilium/ebpf => github.com/ti-mo/ebpf v0.0.0-20200331133758-a258b0c67078

require (
github.com/blang/semver v3.5.1+incompatible
github.com/cilium/ebpf v0.0.0-20200319110858-a7172c01168f
github.com/google/nftables v0.0.0-20191115091743-3ba45f5d7848
github.com/gorilla/mux v1.7.0
github.com/influxdata/influxdb v1.7.4
github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589 // indirect
github.com/iovisor/gobpf v0.0.0-20191129151106-ac26197bb7be // indirect
github.com/jsimonetti/rtnetlink v0.0.0-20191203001355-5d027701a5b7
github.com/lorenzosaino/go-sysctl v0.1.0
github.com/magefile/mage v1.8.0
Expand All @@ -22,10 +24,9 @@ require (
github.com/spf13/cobra v0.0.3
github.com/spf13/viper v1.3.2
github.com/stretchr/testify v1.2.2
github.com/ti-mo/gobpf v0.0.0-20191203213510-34947af25786
github.com/ti-mo/kconfig v0.0.0-20181208153747-0708bf82969f
github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6
golang.org/x/sys v0.0.0-20191029155521-f43be2a4598c
golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9
lukechampine.com/blake3 v0.4.0
)
12 changes: 8 additions & 4 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,6 @@ github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589 h1:oN2MMxbnMD/
github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589/go.mod h1:YVhys+JOY4wmXtJvdtkzLhS2K/r/px/vPc+EAddK+pg=
github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9/go.mod h1:Js0mqiSBE6Ffsg94weZZ2c+v/ciT8QRHFOap7EKDrR0=
github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368/go.mod h1:Wbbw6tYNvwa5dlB6304Sd+82Z3f7PmVZHVKU637d4po=
github.com/iovisor/gobpf v0.0.0-20191129151106-ac26197bb7be h1:WNEVzJxiZ+v09Rr6YWuz4v0lOXJIrGXDZYQDisq64EA=
github.com/iovisor/gobpf v0.0.0-20191129151106-ac26197bb7be/go.mod h1:+5U5qu5UOu8YJ5oHVLvWKH7/Dr5QNHU7mZ2RfPEeXg8=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
github.com/jefferai/jsonx v0.0.0-20160721235117-9cc31c3135ee/go.mod h1:N0t2vlmpe8nyZB5ouIbJQPDSR+mH6oe7xHB9VZHSUzM=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
Expand Down Expand Up @@ -314,8 +312,10 @@ github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/tcnksm/go-input v0.0.0-20180404061846-548a7d7a8ee8/go.mod h1:IlWNj9v/13q7xFbaK4mbyzMNwrZLaWSHx/aibKIZuIg=
github.com/testcontainers/testcontainer-go v0.0.0-20181115231424-8e868ca12c0f/go.mod h1:SrG3IY071gtmZJjGbKO+POJ57a/MMESerYNWt6ZRtKs=
github.com/ti-mo/gobpf v0.0.0-20191203213510-34947af25786 h1:bpDSXOSWhQcOn4FGAakaGNwVZzkifT59l1EJZ8VRl/I=
github.com/ti-mo/gobpf v0.0.0-20191203213510-34947af25786/go.mod h1:uVTtCPNJ7VjhEQFwv7KW+kyfmU92V+XsFlfmdvmt7/U=
github.com/ti-mo/ebpf v0.0.0-20200329172224-d4cb7342e230 h1:7nXADlkIlhuHA3kT+5B/n9CdeGADaOWxr4dSzig/RcY=
github.com/ti-mo/ebpf v0.0.0-20200329172224-d4cb7342e230/go.mod h1:XT+cAw5wfvsodedcijoh1l9cf7v1x9FlFB/3VmF/O8s=
github.com/ti-mo/ebpf v0.0.0-20200331133758-a258b0c67078 h1:nNOESbChqDV9JopTdrYznTrCEDpyO8tnh9V7Naf4C3Y=
github.com/ti-mo/ebpf v0.0.0-20200331133758-a258b0c67078/go.mod h1:XT+cAw5wfvsodedcijoh1l9cf7v1x9FlFB/3VmF/O8s=
github.com/ti-mo/kconfig v0.0.0-20181208153747-0708bf82969f h1:T7sfRIPh8vsabiYlogEpC+ydpskAyqNQMx/tkPsEgi4=
github.com/ti-mo/kconfig v0.0.0-20181208153747-0708bf82969f/go.mod h1:a+7FMqGrlFRrDR6qCcr2uWXFGoJ6iAxn3JqyjYMj2GE=
github.com/tinylib/msgp v1.0.2/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
Expand Down Expand Up @@ -396,6 +396,8 @@ golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191029155521-f43be2a4598c h1:S/FtSvpNLtFBgjTqcKsRpsa6aVsI6iztaz1bQd9BJwE=
golang.org/x/sys v0.0.0-20191029155521-f43be2a4598c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9 h1:1/DFK4b7JH8DmkqhUk48onnSfrPzImPoVxuomtbT2nk=
golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
Expand All @@ -409,6 +411,8 @@ golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gonum.org/v1/gonum v0.0.0-20181121035319-3f7ecaa7e8ca/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo=
gonum.org/v1/netlib v0.0.0-20181029234149-ec6d1f5cefe6/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw=
google.golang.org/api v0.0.0-20181021000519-a2651947f503 h1:UK7/bFlIoP9xre0fwSiXFaZZSpzmaen5MKp1sppNJ9U=
Expand Down
Loading