You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Kirill Shvakov bb163f67f3
Merge pull request #276 from naylinn69/master
2 months ago
examples Change package name 9 months ago
lib Add DateTime64 support 3 months ago
.gitignore Added tests for like queries 1 year ago
.travis.yml Change package name 9 months ago
CONTRIBUTING.md Use Docker Compose. 2 years ago
LICENSE Update LICENSE 6 months ago
Makefile add connection timeout and tls config for client certs 1 year ago
README.md add new connection_open_strategy option's description to README 5 months ago
array.go sync: 1 year ago
bootstrap.go Merge pull request #255 from clobucks/feature/time-random-open-connection-strategy 5 months ago
bootstrap_test.go fix now() function 5 months ago
clickhouse.go Implement driver.ExecerContext 2 months ago
clickhouse_columnar_test.go added block write column function for IP 7 months ago
clickhouse_compress_test.go add lz4 compress support 1 year ago
clickhouse_custom_types_test.go #64 add missing support of driver.Valuer interface 2 years ago
clickhouse_decimal_test.go add more test on nullable & decimal 5 months ago
clickhouse_direct_test.go update ip test 9 months ago
clickhouse_exception.go add type aliases 1 year ago
clickhouse_naive_test.go update readme 3 years ago
clickhouse_negative_test.go new Auth code 4 months ago
clickhouse_nullable_test.go #133 fix 1 year ago
clickhouse_ping.go Change package name 9 months ago
clickhouse_profile_info.go add support Array types for direct interface 3 years ago
clickhouse_progress.go remove support for older versions of ClickHouse 2 years ago
clickhouse_read_block.go Change package name 9 months ago
clickhouse_read_meta.go Change package name 9 months ago
clickhouse_send_query.go Change package name 9 months ago
clickhouse_test.go Add DateTime64 support 3 months ago
clickhouse_write_block.go Change package name 9 months ago
connect.go add connOpenTimeRandom strategy for choosing host to connect to 5 months ago
docker-compose.yml Use Docker Compose. 2 years ago
go.mod Change package name 9 months ago
go.sum Change package name 9 months ago
go.test.sh #58 try codecov.io 3 years ago
helpers.go Fix error expected 1 arguments, got 0 #208 10 months ago
helpers_test.go Fix error expected 1 arguments, got 0 #208 10 months ago
issues_test.go feat: increase number of supported settings 1 year ago
query_settings.go add settings: max_threads 6 months ago
result.go update readme 3 years ago
rows.go implement RowsColumnTypeNullable and RowsColumnTypePrecisionScale interfaces by rows struct to get decimal's precision and scale via func (*Rows) ColumnTypes 5 months ago
stmt.go Change package name 9 months ago
tls_config.go add connection timeout and tls config for client certs 1 year ago
word_matcher.go Added support to pass limit values as bind variables, tests 2 years ago
word_matcher_test.go Added support to pass limit values as bind variables, tests 2 years ago
write_column.go Change package name 9 months ago

README.md

ClickHouse Build Status Go Report Card codecov

Golang SQL database driver for Yandex ClickHouse

Key features

  • Uses native ClickHouse tcp client-server protocol
  • Compatibility with database/sql
  • Round Robin load-balancing
  • Bulk write support : begin->prepare->(in loop exec)->commit
  • LZ4 compression support (default to use pure go lz4, switch to use cgo lz4 by turn clz4 build tags on)

DSN

  • username/password - auth credentials
  • database - select the current default database
  • read_timeout/write_timeout - timeout in second
  • no_delay - disable/enable the Nagle Algorithm for tcp socket (default is 'true' - disable)
  • alt_hosts - comma separated list of single address host for load-balancing
  • connection_open_strategy - random/in_order (default random).
    • random - choose random server from set
    • in_order - first live server is choosen in specified order
    • time_random - choose random(based on current time) server from set. This option differs from random in that randomness is based on current time rather than on amount of previous connections.
  • block_size - maximum rows in block (default is 1000000). If the rows are larger then the data will be split into several blocks to send them to the server. If one block was sent to the server, the data will be persisted on the server disk, we can't rollback the transaction. So always keep in mind that the batch size no larger than the block_size if you want atomic batch insert.
  • pool_size - maximum amount of preallocated byte chunks used in queries (default is 100). Decrease this if you experience memory problems at the expense of more GC pressure and vice versa.
  • debug - enable debug output (boolean value)

SSL/TLS parameters:

  • secure - establish secure connection (default is false)
  • skip_verify - skip certificate verification (default is false)
  • tls_config - name of a TLS config with client certificates, registered using clickhouse.RegisterTLSConfig(); implies secure to be true, unless explicitly specified

example:

tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000

Supported data types

  • UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64
  • Float32, Float64
  • String
  • FixedString(N)
  • Date
  • DateTime
  • IPv4
  • IPv6
  • Enum
  • UUID
  • Nullable(T)
  • Array(T) (one-dimensional) godoc

TODO

  • Support other compression methods(zstd ...)

Install

go get -u github.com/ClickHouse/clickhouse-go

Example

package main

import (
	"database/sql"
	"fmt"
	"log"
	"time"

	"github.com/ClickHouse/clickhouse-go"
)

func main() {
	connect, err := sql.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
	if err != nil {
		log.Fatal(err)
	}
	if err := connect.Ping(); err != nil {
		if exception, ok := err.(*clickhouse.Exception); ok {
			fmt.Printf("[%d] %s \n%s\n", exception.Code, exception.Message, exception.StackTrace)
		} else {
			fmt.Println(err)
		}
		return
	}

	_, err = connect.Exec(`
		CREATE TABLE IF NOT EXISTS example (
			country_code FixedString(2),
			os_id        UInt8,
			browser_id   UInt8,
			categories   Array(Int16),
			action_day   Date,
			action_time  DateTime
		) engine=Memory
	`)

	if err != nil {
		log.Fatal(err)
	}
	var (
		tx, _   = connect.Begin()
		stmt, _ = tx.Prepare("INSERT INTO example (country_code, os_id, browser_id, categories, action_day, action_time) VALUES (?, ?, ?, ?, ?, ?)")
	)
	defer stmt.Close()

	for i := 0; i < 100; i++ {
		if _, err := stmt.Exec(
			"RU",
			10+i,
			100+i,
			clickhouse.Array([]int16{1, 2, 3}),
			time.Now(),
			time.Now(),
		); err != nil {
			log.Fatal(err)
		}
	}

	if err := tx.Commit(); err != nil {
		log.Fatal(err)
	}

	rows, err := connect.Query("SELECT country_code, os_id, browser_id, categories, action_day, action_time FROM example")
	if err != nil {
		log.Fatal(err)
	}
	defer rows.Close()

	for rows.Next() {
		var (
			country               string
			os, browser           uint8
			categories            []int16
			actionDay, actionTime time.Time
		)
		if err := rows.Scan(&country, &os, &browser, &categories, &actionDay, &actionTime); err != nil {
			log.Fatal(err)
		}
		log.Printf("country: %s, os: %d, browser: %d, categories: %v, action_day: %s, action_time: %s", country, os, browser, categories, actionDay, actionTime)
	}

	if err := rows.Err(); err != nil {
		log.Fatal(err)
	}

	if _, err := connect.Exec("DROP TABLE example"); err != nil {
		log.Fatal(err)
	}
}

Use sqlx

package main

import (
	"log"
	"time"

	"github.com/jmoiron/sqlx"
	_ "github.com/ClickHouse/clickhouse-go"
)

func main() {
	connect, err := sqlx.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
	if err != nil {
		log.Fatal(err)
	}
	var items []struct {
		CountryCode string    `db:"country_code"`
		OsID        uint8     `db:"os_id"`
		BrowserID   uint8     `db:"browser_id"`
		Categories  []int16   `db:"categories"`
		ActionTime  time.Time `db:"action_time"`
	}

	if err := connect.Select(&items, "SELECT country_code, os_id, browser_id, categories, action_time FROM example"); err != nil {
		log.Fatal(err)
	}

	for _, item := range items {
		log.Printf("country: %s, os: %d, browser: %d, categories: %v, action_time: %s", item.CountryCode, item.OsID, item.BrowserID, item.Categories, item.ActionTime)
	}
}