Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support set map value by key and fix dependency #43

Open
wants to merge 43 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
14bc2eb
[dep] Init glide https://github.com/pingginp/cqlc/issues/1
pingginp Sep 15, 2018
490ad34
[dep] Fix gocql ColumnClusterKey etc.
pingginp Sep 15, 2018
c0a9108
[gen] Format generated columns.go
pingginp Sep 15, 2018
939cc7b
[test] Bump go and cassandra version in Travis
pingginp Sep 15, 2018
3c41034
[gen] Remove go-bindata and use template directly
pingginp Sep 15, 2018
d524090
[test] Install glide to $GOPATH/bin
pingginp Sep 15, 2018
2a3fdb1
[test] Move project to right go path
pingginp Sep 15, 2018
38ead15
[test] Use C* 2.2.1
pingginp Sep 15, 2018
c65af03
[test] Remove bindata rule from test deps
pingginp Sep 15, 2018
e10a96c
Fix Printf format error
pingginp Sep 15, 2018
9d2aff5
[test] go get github.com/gocql/cql for generator
pingginp Sep 15, 2018
eca906a
[test] Only test cqlc, remove generator
pingginp Sep 15, 2018
69de726
[gen] Add set map value by key #2
pingginp Sep 15, 2018
361e785
[test] Can't connect to C* using docker on Mac #6
pingginp Sep 15, 2018
d623f38
Fix #6 set broadcast_addr and correct port
pingginp Sep 15, 2018
3df5870
[gen] C*3 seems to be broken
pingginp Sep 15, 2018
69d2188
[gen] Set Map value didn't work ...
pingginp Sep 15, 2018
6b9a8cc
[gen] Bind value for map didn't work #2
pingginp Sep 15, 2018
faf7b6a
[gen] Flatten binding in BuildStatement #2 works
pingginp Sep 15, 2018
30cbe8e
[dep] Remove seelog add pkg/errors
pingginp Sep 15, 2018
11e8efc
[gen] Use separated type for map key value
pingginp Sep 15, 2018
52941fa
[test] Run gocql and cqlc in parallel
pingginp Sep 15, 2018
3cff150
[test] Fix #3 use docker for e2e test
pingginp Sep 15, 2018
ff73b30
[gen] Change generated header to canonical form
pingginp Sep 15, 2018
42d844c
Merge pull request #4 from pingginp/reboot
pingginp Sep 15, 2018
e6dfa5d
[runtime] Use logrus instead of stdlog
pingginp Sep 16, 2018
d457ca9
[gen] Replace log.Fatalf w/ errors.Errorf
pingginp Sep 16, 2018
43a4e3a
[runtime] Update version defined in runtime
pingginp Sep 16, 2018
d426ffc
Merge pull request #9 from pingginp/logrus
pingginp Sep 16, 2018
f90e949
Fix C*3 https://github.com/pingginp/cqlc/issues/7
pingginp Feb 18, 2019
2760cf7
Merge pull request #12 from pingginp/c3
pingginp Feb 18, 2019
1893983
[dep] Remove logrus and use interface Fix #11
pingginp Feb 18, 2019
f33f233
[dep] Init go mod support Fix #10
pingginp Feb 18, 2019
fda8beb
[release] 0.12.0
pingginp Feb 18, 2019
1e4e23e
[doc] Change not support C3 to support
pingginp Feb 18, 2019
33f9519
[doc] Add go mod instruction
pingginp Feb 18, 2019
7fc6fe0
[gen] #7 forgot to update column type mapping
pingginp Feb 19, 2019
1de1d77
[release] 0.12.1
pingginp Feb 19, 2019
2684311
[runtime] Support if in delete statement #13
pingginp Mar 10, 2019
bb26635
[gen] Allow Eq on all columns for DELTE IF #13
pingginp Mar 10, 2019
1910b32
[doc] Update change log
pingginp Mar 10, 2019
292abab
Merge pull request #14 from pingginp/runtime/delete-if
pingginp Mar 10, 2019
fc04d23
[release] 0.13.0
pingginp Mar 10, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
output/
test/.fixtures/collections/input.go
test/collections.cql

.idea
vendor
51 changes: 29 additions & 22 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,40 +1,47 @@
sudo: required
language: go

sudo: false

cache:
directories:
- $HOME/.ccm/repository
services:
- docker

matrix:
fast_finish: true

env:
global:
# TODO: why need this?
- GOMAXPROCS=2
- DOCKER_COMPOSE_VERSION=1.22.0
matrix:
- CASS=2.1.9
- CASS=2.2.1
# - CASS=3.11

go:
- 1.4
- 1.5
- "1.10"
# - "1.11"
# - "tip"

before_install:
- go get github.com/stretchr/testify/assert
- go get github.com/jteeuwen/go-bindata/...
- go get github.com/relops/csvb
- go get github.com/cihub/seelog
- export PATH=$PATH:/home/travis/gopath/bin
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
- curl -sSL https://github.com/Masterminds/glide/releases/download/v0.13.1/glide-v0.13.1-linux-amd64.tar.gz | tar -vxz -C ${GOPATH}/bin --strip=1
# FIXME: workaround for gopath
- mkdir -p ${GOPATH}/src/github.com/relops/cqlc
- cp -r ./. ${GOPATH}/src/github.com/relops/cqlc
- cd ${GOPATH}/src/github.com/relops/cqlc

install:
- pip install --user PyYAML six
- git clone https://github.com/pcmanus/ccm.git
- pushd ccm
- ./setup.py install --user
- popd
- go get -v ./...
# - pip install --user PyYAML six
# - git clone https://github.com/pcmanus/ccm.git
# - pushd ccm
# - ./setup.py install --user
# - popd
- glide install
- make install

script:
- set -e
- PATH=$PATH:$HOME/.local/bin bash -x integration.sh $CASS
- make test
# e2e
- make travis-test

38 changes: 38 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Changelog

NOTE: this file format is based on [gaocegege/maintainer](https://github.com/gaocegege/maintainer/blob/master/CHANGELOG.md)

## Unreleased

## 0.13.0 (2019-03-10)

Was going to make it 0.12.2 but since it breaks both runtime and generated code, bump minor version number

- support `IF` in `DELETE` [#13](https://github.com/pingginp/cqlc/issues/13)
- in generated column bindings allow `Eq` on all columns, previously only primary key, index are allowed, which blocks using
other columns in condition queries after `If`

## 0.12.1 (2019-02-18)

- previous release didn't update all the mapping in generator

## 0.12.0 (2019-02-18)

[Closed issues](https://github.com/pingginp/cqlc/issues?q=is%3Aissue+milestone%3A0.12.0+is%3Aclosed)

Merged pull requests

- [#12](https://github.com/pingginp/cqlc/pull/12) one line fix to support Cassandra 3

## 0.11.0 (2018-09-15)

[Closed issues](https://github.com/pingginp/cqlc/issues?q=is%3Aissue+is%3Aclosed+milestone%3A0.11.0)

Merged pull requests

- Reboot [#4](https://github.com/pingginp/cqlc/pull/4) the project now compiles and support set map value by key
- [#9](https://github.com/pingginp/cqlc/pull/9) remove `log.Fatal` and use logrus

## 0.10.5 (2015-09-10)

The last commit in upstream https://github.com/relops/cqlc/commit/9427a2081fb4f4910b0af8fc80d09c109b4f9815
71 changes: 47 additions & 24 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,35 +1,58 @@
CCM_NODE ?= node1
CQLSH_CMD ?= ccm $(CCM_NODE) cqlsh
VERSION = 0.13.0
LDFLAGS = -X main.Version=$(VERSION)
GO = CGO_ENABLED=0 go
GO_LINUX = GOOS=linux GOARCH=amd64 $(GO)
GO_MAC = GOOS=darwin GOARCH=amd64 $(GO)
GO_WINDOWS = GOOS=windows GOARCH=amd64 $(GO)

test/collections.cql: test/tmpl/schema.tmpl test/schema_generator.go
cd test; go run schema_generator.go
.PHONY: gen fmt build install test

test/.fixtures/collections/input.go: test/tmpl/input.tmpl test/schema_generator.go
cd test; go run schema_generator.go
fmt:
gofmt -d -l -w cqlc generator e2e

schema: test/collections.cql
-$(CQLSH_CMD) -f test/keyspace.cql
$(CQLSH_CMD) -k cqlc -f test/schema.cql
$(CQLSH_CMD) -k cqlc -f test/collections.cql
$(CQLSH_CMD) -k cqlc -f test/shared.cql
$(CQLSH_CMD) -k cqlc2 -f test/shared.cql
build:
$(GO) build -ldflags "$(LDFLAGS)" -o build/cqlc .

cqlc/columns.go: cqlc/tmpl/columns.tmpl cqlc/column_generator.go
cd cqlc; go run column_generator.go
build-all: build-linux build-mac build-windows

build-linux:
$(GO_LINUX) build -ldflags "$(LDFLAGS)" -o build/cqlc-linux .

build-mac:
$(GO_MAC) build -ldflags "$(LDFLAGS)" -o build/cqlc-mac .

columns: cqlc/columns.go
build-windows:
$(GO_WINDOWS) build -ldflags "$(LDFLAGS)" -o build/cqlc-windows .

bindata: generator/binding_tmpl.go
install:
go install -ldflags "$(LDFLAGS)" .

input: test/.fixtures/collections/input.go test/collections.cql
# sync the version defined in runtime with Makefile
update-ver:
# NOTE: mac's default sed is not GNU sed https://stackoverflow.com/questions/4247068/sed-command-with-i-option-failing-on-mac-but-works-on-linux
sed -i .bak -E 's/const Version = "(.*)"/const Version = "$(VERSION)"/g' cqlc/ver.go

release: update-ver build-all
cd build; rm -f *.zip
cd build; zip cqlc-$(VERSION)-linux.zip cqlc-linux
cd build; zip cqlc-$(VERSION)-mac.zip cqlc-mac
cd build; zip cqlc-$(VERSION)-windows.zip cqlc-windows

# generate highly duplicated part in runtime
gen:
cd cqlc; go run column_generator.go

generator/binding_tmpl.go: generator/tmpl/binding.tmpl
go-bindata -pkg=generator -o=generator/binding_tmpl.go generator/tmpl
test: test-unit

test: columns bindata schema test/.fixtures/collections/input.go
go test -v ./...
test-unit:
go test -v ./cqlc

format:
gofmt -w cqlc generator integration test
travis-test: install
docker-compose -f e2e/docker-compose.yaml up -d c2
./wait-on-c.sh
docker ps
sleep 5
go test -v ./e2e

.PHONY: test columns bindata
travis-tear:
cd e2e && make down
74 changes: 66 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,71 @@
cqlc
----
# cqlc

[![Join the chat at https://gitter.im/relops/cqlc](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/relops/cqlc?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![Build Status](https://travis-ci.org/pingginp/cqlc.svg?branch=master)](https://travis-ci.org/pingginp/cqlc)

[![Build Status](https://travis-ci.org/relops/cqlc.png?branch=master)](https://travis-ci.org/relops/cqlc)
[![GoDoc](http://godoc.org/_?status.png)](http://godoc.org/github.com/relops/cqlc/cqlc)
This a fork of [relops/cqlc](https://github.com/relops/cqlc) the upstream is no longer maintained.

`cqlc` generates Go code from your Cassandra schema so that you can write type safe CQL statements in Go with a natural query syntax.
## Usage

For more details please visit [http://relops.com/cqlc][cqlc]
````bash
# install the generator to $GOPATH/bin
make install
# generate table and column definition based on schema in keyspace cqlc
cqlc --instance=127.0.0.1 --keyspace=cqlc --package=foo --output=foo.go --symbols
````

[cqlc]: http://relops.com/cqlc
You need to change the repo path in `glide.yaml` to use this fork

````yaml
- package: github.com/relops/cqlc
version: master
repo: https://github.com/pingginp/cqlc.git
````

If you use `go mod`, add the following in your `go.mod`, [go mod wiki](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive)

````text
replace github.com/relops/cqlc => github.com/pingginp/cqlc v0.12.0
````

## Dev

- clone the repo to `$GOPATH/src/github.com/relops/cqlc`

````bash
# generate columns
make cqlc/columns.go
# e2e test
# TODO: you need to run it twice if schema changed because first time it will generate package based on schema, which won't get compiled ...
# this same as when using latex ... you do xelatex several times when there is bib ...
make travis-test
make travis-tear
# release, update cqlc/ver.go, build and zip binary for all three platforms, only mac is tested
make release
````

The code has two part, runtime and generator

- [cqlc](cqlc) is the runtime, a query builder, don't get mislead by the [column_generator.go](cqlc/column_generator.go)
it is mainly for generating runtime code that ships with the library
- [generator](generator) generates table and column definition based on schema, NOTE: it now [supports Cassandra 3](https://github.com/pingginp/cqlc/issues/7)

### Runtime

The main modification to the runtime are listed below

- [support update map value by key](doc/set-map-value-by-key.md), previously, cqlc can only update entire map. (This change only requires update runtime)
- support `IF` in `DELETE` [#13](https://github.com/pingginp/cqlc/issues/13)

### Generator

The main modification to the generator are listed below

- generator now compiles, caused by breaking change of constant name in gocql
- support Cassandra 3 by adding a new literalType mapping for text -> string [#12](https://github.com/pingginp/cqlc/pull/12)
- allow `Eq` on all columns to support `IF` in `DELETE`

The overall generator logic is

- get table meta using gocql
- render the template defined in `tmpl.go` using template helper methods defined in `template.go`
- `valueType` is returning empty value for `text`, just add a new mapping in `literalTypes` fixed this [#7](https://github.com/pingginp/cqlc/issues/7)
2 changes: 2 additions & 0 deletions build/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
cqlc
cqlc-*
1 change: 1 addition & 0 deletions cqlc/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.bak
46 changes: 17 additions & 29 deletions cqlc/column_generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,12 @@ package main

import (
"bytes"
"fmt"
log "github.com/cihub/seelog"
"go/format"
"io/ioutil"
"log"
"text/template"
)

var logConfig = `
<seelog type="sync">
<outputs formatid="main">
<console/>
</outputs>
<formats>
<format id="main" format="%Date(2006-02-01 03:04:05.000) - %Msg%n"/>
</formats>
</seelog>`

func init() {
logger, err := log.LoggerFromConfigAsString(logConfig)

if err != nil {
fmt.Printf("Could not load seelog configuration: %s\n", err)
return
}

log.ReplaceLogger(logger)
}

type TypeInfo struct {
Prefix string
Literal string
Expand All @@ -57,17 +36,26 @@ func main() {

t, err := template.New("columns.tmpl").ParseFiles("tmpl/columns.tmpl")
if err != nil {
log.Errorf("Could not open template: %s", err)
log.Fatalf("Could not open template: %s", err)
return
}

var buf bytes.Buffer
if err := t.Execute(&buf, params); err != nil {
log.Fatalf("Could not render template: %s", err)
return
}

var b bytes.Buffer
t.Execute(&b, params)
b, err := format.Source(buf.Bytes())
if err != nil {
log.Fatalf("Could not format rendered template as go code: %s", err)
return
}

if err := ioutil.WriteFile("columns.go", b.Bytes(), 0644); err != nil {
log.Errorf("Could not write templated file: %s", err)
if err := ioutil.WriteFile("columns.go", b, 0644); err != nil {
log.Fatalf("Could not write templated file: %s", err)
return
}

log.Info("Regenerated columns")
log.Println("Regenerated columns")
}
Loading