Commit Graph

172 Commits

Author SHA1 Message Date
Tomasz Kleczek
42d61191c4 storage: conformance tests improvements 2019-09-27 13:54:54 +02:00
Nandor Kracser
d2c33db8a8 storage/mysql: support pre-5.7.20 instances with tx_isolation only 2019-09-23 09:36:01 +02:00
Michael Venezia
395febf808
storage/kubernetes: Removing Kubernetes TPR support
Third Party Resources (TPR) have been removed from Kubernetes for
roughly 2 years.  This commit removes the support dex had for them.

Documentation has been updated to reflect this and to instruct users
on how to migrate from TPR-powered dex environment to a Custom Resource
Defintion (CRD) based one that dex > v2.17 will support
2019-08-14 09:28:18 -04:00
Stephan Renatus
d9487e553b
*: fix some lint issues
Mostly gathered these using golangci-lint's deadcode and ineffassign
linters.

Signed-off-by: Stephan Renatus <srenatus@chef.io>
2019-07-30 11:29:08 +02:00
Nandor Kracser
a572ad8fec storage/sql: rework of the original MySQL PR 2019-07-23 14:27:10 +02:00
Pavel Borzenkov
e53bdfabb9 storage/sql: initial MySQL storage implementation
It will be shared by both Postgres and MySQL configs.

Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>
2019-07-23 14:26:21 +02:00
Mark Sagi-Kazar
d877fca092
Fix coding style 2019-02-22 21:43:55 +01:00
Mark Sagi-Kazar
06521ffa49
Remove the logrus logger wrapper 2019-02-22 21:31:46 +01:00
Mark Sagi-Kazar
be581fa7ff
Add logger interface and stop relying on Logrus directly 2019-02-22 13:38:57 +01:00
Haines Chan
b78b8aeee0 Replace "GET", "POST" to http.MethodGet and http.MethodPost 2018-12-27 16:27:36 +08:00
Stephan Renatus
73fdf4f75b
storage/sql/postgres: expose stdlib tunables, set them for tests
- adapted TestUnmarshalConfig to ensure the fields are read in
- added a test to see that at least MaxOpenConns works:
  - this is only exposed through (*db).Stats() in go 1.11, so this test
    has a build tag
  - the other two configurables can't be read back, so we've got to
    trust that the mechanism works given the one instance that's tested..

Signed-off-by: Stephan Renatus <srenatus@chef.io>
2018-11-30 09:55:01 +01:00
Stephan Renatus
8f3cca7ba4
Revert "retry on serialization errors" 2018-11-29 08:24:13 +01:00
Stephan Renatus
5f054fcf2e
Merge pull request #1342 from concourse/pr/retry-on-pg-serialization-error
retry on serialization errors
2018-11-21 10:29:46 +01:00
Alex Suraci
85dd0684ba extract and document serialization failure check 2018-11-20 10:50:55 -05:00
Alex Suraci
587081a643 postgres: refactor error handling to fix retrying
prior to this change, many of the functions in the ExecTx callback would
wrap the error before returning it. this made it impossible to check
for the error code.

instead, the error wrapping has been moved to be external to the
`ExecTx` callback, so that the error code can be checked and
serialization failures can be retried.
2018-11-20 10:50:55 -05:00
Alex Suraci
aa068b667a postgres: improve readability of error check 2018-11-20 10:50:55 -05:00
Alex Suraci
9b9013a560 postgres: use stdlib to set serializable tx level
also use a context for the rollback, which is a bit cleaner since it
only results in one 'defer', rather than N from the loop
2018-11-20 10:50:55 -05:00
Alex Suraci
7e96021428 retry on serialization errors 2018-11-20 10:50:55 -05:00
Stephan Renatus
6182f213ef
storage/conformance: remove old build tags
Signed-off-by: Stephan Renatus <srenatus@chef.io>
2018-11-20 16:41:13 +01:00
Stephan Renatus
0740c2370d
storage/etcd: remove standup.sh
Signed-off-by: Stephan Renatus <srenatus@chef.io>
2018-11-20 16:41:12 +01:00
Stephan Renatus
1d0568efe9
storage/sql: remove standup.sh
Signed-off-by: Stephan Renatus <srenatus@chef.io>
2018-11-20 16:41:12 +01:00
Alex Suraci
0b856d1a75 pg: support host:port for backwards-compatibility 2018-11-20 10:22:39 -05:00
Stephan Renatus
b1fd2fa8b1
Merge pull request #1346 from concourse/pr/postgres-unix-sockets
Use pq connection parameters instead of URLs for postgres connections

This enables the use of socket paths like /var/run/postgresql for the 'host' instead of requiring TCP. Also, we know allow using a non-default port.
2018-11-20 15:52:40 +01:00
Divya Dadlani
f82b904d05 postgres: use connection string instead of url
otherwise it's impossible to use a Unix socket, as the path gets escaped
awkwardly.

Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
2018-11-20 09:32:44 -05:00
Alex Suraci
7c63be4104 remove incomplete mysql and cockroachdb support 2018-11-16 18:07:20 +00:00
Alex Suraci
dcca427592 fix bogus conformance failure due to time zones
this failed on my machine due to the unexported 'loc' field of the time
structure - it was nil in one and set to a ton of tiemzone data in the
other. instead let's just compare the unix timestamp value and zero it
out for the struct comparison.
2018-11-15 13:51:07 -05:00
Song.Jin
5f0a03a06b modify log msg as per suggested 2018-11-06 11:18:55 +11:00
Song.Jin
9b5bec1ddf check if crd exist before try creating them 2018-11-04 11:43:28 +11:00
Song.Jin
d2daa4e2ac allow it to disable CRD creation 2018-11-02 21:13:37 +11:00
Stephan Renatus
666356d22d
Merge pull request #1266 from byxorna/gabe/fix-etcd-timeout-bug
fix timeout bug for etcd3 client connect
2018-09-10 10:36:38 +02:00
Stephan Renatus
b9f6594bf0 *: github.com/coreos/dex -> github.com/dexidp/dex
Signed-off-by: Stephan Renatus <srenatus@chef.io>
2018-09-05 17:57:08 +02:00
Gabe Conradi
94bd948aac fix timeout bug for etcd3 client connect 2018-08-02 17:41:38 -04:00
Eric Chiang
ccf85a7269
Merge pull request #1108 from dqminh/etcd-storage
Add etcd backed storage
2017-11-06 08:36:43 -08:00
Daniel Dao
e617197871 storage/etcd: document struct tag in code
This explicitly adds struct tags for etcd storage instead of implicitly
depends on yaml/json config serialization.
2017-11-06 14:46:18 +00:00
rithu leena john
42ef8fd802
Merge pull request #1072 from ericchiang/k8s-test
*: run kubernetes tests in travis
2017-10-31 10:34:26 -07:00
Eric Chiang
3d2d92b31b *: run kubernetes tests in travis 2017-10-31 10:29:52 -07:00
Daniel Dao
ca114f7812 storage: add etcd storage
This patch adds etcd storage implementation. This should be useful in
environments where
- we dont want to depends on a separate, hard to maintain SQL cluster
- we dont want to incur the overhead of talking to kubernetes apiservers
- kubernetes is not available yet, or if kubernetes depends on dex
to perform authentication and the operator would like to remove any
circular dependency if possible.
2017-10-31 14:43:13 +00:00
Daniel Dao
2b13bdd12d
storage: fix list connector test
The previous test doesnt actually testing ListConnectors code. For
example the following pseudocode will pass the test:

```
ListConnectors() { return nil, nil }
```

Instead change to actually fetch and compare list of connectors,
ordering by name
2017-10-27 15:26:05 +01:00
Eric Chiang
0aabf2d1ea Merge pull request #1085 from rphillips/fixes/http_client_timeout
add client request timeout
2017-09-27 13:28:13 -07:00
Ryan Phillips
0318cd99b0 add client request timeout and dialer deadline 2017-09-26 18:52:11 -05:00
Chance Zibolski
9d7b0b59bd storage/kubernetes: Log before registering custom resources
Logging before attempting to make any connection to Kubernetes is useful when the connection hangs and dex is killed before it can log any errors.
2017-09-26 16:23:49 -07:00
rithu john
d2706fcab8 storage/kubernetes: Correct the OfflineSession object CRD definition 2017-09-19 14:58:42 -07:00
rithu john
1311caf864 storage/kubernetes: add CRD support 2017-09-14 11:48:17 -07:00
rithu john
146481375e [WIP]: add CRD support 2017-09-13 10:57:54 -07:00
rithu john
fd4f57b5f3 storage/static.go: storage backend should not explicitly lower-case email ids. 2017-08-24 15:50:32 -07:00
Eric Stroczynski
4a88d0641a : update {S->s}irupsen/logrus 2017-07-25 13:46:44 -07:00
Kazumasa Kohtaka
cc314690f4 Avoid generating an identifer which starts with a number because it may be used as a SAML's ID attribute 2017-07-06 21:28:48 +09:00
rithu john
8c9c2518f5 server: account for dynamically changing connector object in storage. 2017-04-25 09:19:02 -07:00
rithu john
5abb4b3df6 storage/static.go: correct the error message that gets displayed. 2017-03-29 11:32:02 -07:00
rithu john
bc55b86d0d storage: add connector object to backend storage. 2017-03-28 14:12:38 -07:00
Eric Chiang
95d237003a Merge pull request #855 from ericchiang/static-storage-fallthrough
storage: make static storages query real storages for some actions
2017-03-20 10:42:34 -07:00
Eric Chiang
4c39bc20ae storage: make static storages query real storages for some actions
If dex is configured with static passwords or clients, let the API
still add or modify objects in the backing storage, so long as
their IDs don't conflict with the static ones. List options now
aggregate resources from the static list and backing storage.
2017-03-20 09:39:38 -07:00
rithu john
9e88924577 storage/conformance: update conformance tests with multiple entries per resource 2017-03-16 16:35:51 -07:00
Eric Chiang
6cb38604d9 storage/kubernetes: log INFO level if TPR already exists, not ERROR 2017-03-15 10:30:10 -07:00
Eric Chiang
0481fccd76 storage/sql: add missing WHERE statement to refresh token update 2017-03-13 15:53:28 -07:00
Eric Chiang
777eeafabc *: update go-oidc and use standard library's context package 2017-03-08 10:33:19 -08:00
Eric Chiang
38c77e0f33 storage/kubernetes: enable HTTP/2 support 2017-02-28 12:42:06 -08:00
Eric Chiang
a7b8e52b92 storage/kubernetes: fix conflict error detection in TRP creation
PR #815 fixed the Kubernetes storage implementation by correctly
returning storage.ErrAlreadyExists on POST conflicts. This caused a
regression in TPR creation (#822) when some, but not all, of the
resources already existed. E.g. for users upgrading from old
versions of dex.

Fixes #822
2017-02-27 11:01:47 -08:00
Eric Chiang
1da2ae279c storage/kubernetes: fix hash initialization bug 2017-02-24 12:55:04 -08:00
Eric Chiang
4be029c6c1 storage/kubernetes: fix kubernetes storage conformance test failures 2017-02-23 19:23:19 -08:00
rithu john
3df1db1864 storage: Surface "already exists" errors. 2017-02-21 15:00:22 -08:00
rithu john
d928ac0677 storage: Add OfflineSession object to backend storage. 2017-02-09 19:01:28 -08:00
Eric Chiang
1eda382789 server: add at_hash claim support
The "at_hash" claim, which provides hash verification for the
"access_token," is a required claim for implicit and hybrid flow
requests. Previously we did not include it (against spec). This
PR implements the "at_hash" logic and adds the claim to all
responses.

As a cleanup, it also moves some JOSE signing logic out of the
storage package and into the server package.

For details see:

https://openid.net/specs/openid-connect-core-1_0.html#ImplicitIDToken
2017-01-13 10:05:24 -08:00
Eric Chiang
312ca7491e storage: add extra fields to refresh token and update method 2017-01-11 12:07:48 -08:00
rithu john
2c03693972 storage: Add ConnectorData to storage RefreshToken. 2017-01-09 15:01:29 -08:00
Eric Chiang
fd20b213bb storage: fix postgres timezone handling
Dex's Postgres client currently uses the `timestamp` datatype for
storing times. This lops of timezones with no conversion, causing
times to lose locality information.

We could convert all times to UTC before storing them, but this is
a backward incompatible change for upgrades, since the new version
of dex would still be reading times from the database with no
locality.

Because of this intrinsic issue that current Postgres users don't
save any timezone data, we chose to treat any existing installation
as corrupted and change the datatype used for times to `timestamptz`.
This is a breaking change, but it seems hard to offer an
alternative that's both correct and backward compatible.

Additionally, an internal flag has been added to SQL flavors,
`supportsTimezones`. This allows us to handle SQLite3, which doesn't
support timezones, while still storing timezones in other flavors.
Flavors that don't support timezones are explicitly converted to
UTC.
2016-12-16 11:46:49 -08:00
Eric Chiang
91cc94dd8f Merge pull request #740 from ericchiang/fix-comment-typos
*: fix comment typos and add go report card icon
2016-12-13 13:17:50 -08:00
Eric Chiang
fe196864c0 *: fix comment typos found with github.com/client9/misspell 2016-12-13 12:23:16 -08:00
rithu john
9949a1313c server: modify error messages to use logrus. 2016-12-13 11:52:44 -08:00
rithu john
2e22a948cf cmd/dex: add logging config and serve logger for different modules. 2016-12-12 15:56:50 -08:00
Eric Chiang
879e018f74 storage/kubernetes: remove unused method 2016-12-08 16:42:18 -08:00
rithu john
ee9738d663 api: adding a gRPC call for listing passwords. 2016-11-17 16:56:54 -08:00
rithu john
19c22807a7 api: adding ListPasswords() method to the storage interface. 2016-11-16 17:25:38 -08:00
Eric Chiang
aa7f304bc1 *: switch to github.com/ghodss/yaml for more consistent YAML parsing
ghodss/yaml converts from YAML to JSON before attempting to unmarshal.
This allows us to:

* Get the correct behavor when decoding base64'd []byte slices.
* Use *json.RawMessage.
* Not have to support extravagant YAML features.
* Let our structs use `json:` tags
2016-11-03 14:39:32 -07:00
Eric Chiang
90e613b328 Merge pull request #649 from rithujohn191/gRPC-endpoints
api: add gRPC endpoints for creating, updating and deleting passwords
2016-11-01 14:20:31 -07:00
rithu leena john
ed7e943406 api: add gRPC endpoints for creating, updating and deleting passwords 2016-11-01 14:10:35 -07:00
Eric Chiang
786e12b15e storage/conformance: expand transaction test suite 2016-10-31 23:01:31 -07:00
Eric Chiang
52e2a1668c storage/sql: use isolation level "serializable" for transactions 2016-10-31 23:00:55 -07:00
Eric Chiang
c0aa63ac97 storage: update godocs 2016-10-28 13:00:13 -07:00
Eric Chiang
d7a75c5b5d storage/kubernetes: allow arbitrary client IDs
Use a hash algorithm to match client IDs to Kubernetes object names.
Because cryptographic hash algorithms produce sums larger than a
Kubernetes name can fit, a non-cryptographic hash is used instead.
Hash collisions are checked and result in errors.
2016-10-27 16:37:58 -07:00
Eric Chiang
4ab78d0ded storage/kubernetes: run transactional conformance tests 2016-10-26 13:30:45 -07:00
Eric Chiang
5720ecf412 storage/conformance: add tests for transactional guarantees 2016-10-26 13:30:45 -07:00
Eric Chiang
d350938fb0 Merge pull request #626 from ericchiang/storage-kubernetes-guess-namespace-from-service-account-token
storage/kubernetes: guess namespace from the service account token
2016-10-25 16:54:58 -07:00
Eric Chiang
101a2bc22a Merge pull request #634 from rithujohn191/kubeconfig_context
storage/kubernetes: set CurrentContext when the Kubeconfig file contains only one context
2016-10-25 14:57:57 -07:00
Eric Chiang
e0b83af981 Merge pull request #629 from ericchiang/dev-storage-kubernetes-dont-print-error
storage/kubernetes: don't automatically print errors on bad HTTP status codes
2016-10-25 14:16:32 -07:00
rithu leena john
9de16f2c45 storage/kubernetes: set CurrentContext when the Kubeconfig file contains only one context 2016-10-25 11:59:34 -07:00
Eric Chiang
bc16de0b58 storage/kubernetes: don't guess the kubeconfig location and change test env
Using the default KUBECONFIG environment variable to indicate that
the Kubernetes tests should be run lead to cases where developers
accidentally ran the tests. This has now been changed to
"DEX_KUBECONFIG" and documentation hsa been added detailing how to
run these tests.

Additionally, no other storage reads environment variables for its
normal configuration (outside of tests) so the Kubernetes storage
no longer does.

Overall, be less surprising.
2016-10-23 20:53:29 -07:00
Eric Chiang
138ff96c00 storage/kubernetes: don't automatically print errors on bad HTTP status codes
These status codes spam the error logs for events like key rotation
and third party resource creation. In these cases "bad" status codes
are expected and shouldn't be automatically printed.
2016-10-23 07:42:42 -07:00
Eric Chiang
8c9c5160b6 storage/kubernetes: guess namespace from the service account token
The in cluster kubernetes client currently requires using the
downward API to determine its namespace. However this value can be
determine by inspecting the service account token mounted into the
pod. As a fallback, use this to guess the current namespace.
2016-10-21 23:35:17 -07:00
Eric Chiang
fe320c1928 storage/sql: fix typo in keys query 2016-10-14 12:28:49 -07:00
Eric Chiang
0a3aabc8ff storage/conformace: add conformance tests for keys 2016-10-14 12:28:49 -07:00
Eric Chiang
691476b477 storage/kubernetes: manage third party resources and drop support for 1.3 2016-10-13 17:41:52 -07:00
Eric Chiang
449f34ed2a storage/sql: print error before calling t.Fatal 2016-10-12 22:00:08 -07:00
Eric Chiang
558059ee58 storage/kubernetes: add garbage collection method 2016-10-12 18:48:23 -07:00
Eric Chiang
9ce05ecf73 storage/sql: add garbage collection method 2016-10-12 18:48:09 -07:00
Eric Chiang
c14ab3c44e storage/memory: add garbage collection method 2016-10-12 18:47:47 -07:00
Eric Chiang
d27f5e411f storage/conformance: add garbage collection tests 2016-10-12 18:47:15 -07:00
Eric Chiang
df6cfa0b7a storage: add GC method to interface to standardize handling 2016-10-12 18:46:10 -07:00
Eric Chiang
2909929b17 *: add the ability to define passwords statically 2016-10-06 10:35:54 -07:00
Eric Chiang
7ff3ce85a2 *: add password resource to kubernetes storage implementation 2016-10-05 17:26:41 -07:00