k-space/dex
Archived
10
0
Fork 0
Code Issues Pull Requests Packages Projects Activity

storage/kubernetes: Removing Kubernetes TPR support

Browse Source 
Third Party Resources (TPR) have been removed from Kubernetes for
roughly 2 years.  This commit removes the support dex had for them.

Documentation has been updated to reflect this and to instruct users
on how to migrate from TPR-powered dex environment to a Custom Resource
Defintion (CRD) based one that dex > v2.17 will support
This commit is contained in:
Michael Venezia 2019-08-13 17:48:37 -04:00
parent d9f6ab4a68
commit 395febf808
No known key found for this signature in database
GPG Key ID: 8C8007B3597AB5EE
6 changed files with 23 additions and 261 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

121
Documentation/storage.md
View File

@ -115,92 +115,16 @@ subjects:
``` ```
## DEPRECATED: Kubernetes third party resources(TPRs) ### Removed: Kubernetes third party resources(TPRs)
__NOTE:__ TPRs are deprecated as of Kubernetes version 1.8. TPR support in dex has been removed. The last version to support TPR
is [v2.17.0](https://github.com/dexidp/dex/tree/v2.17.0)
The default behavior of dex from release v2.7.0 onwards is to utilize CRDs to manage its custom resources. If users would like to use dex with a Kubernetes version lower than 1.7, they will have to force dex to use TPRs instead of CRDs. If you are currently running dex using TPRs, you will need to [migrate to CRDs](https://github.com/dexidp/dex/blob/v2.17.0/Documentation/storage.md#migrating-from-tprs-to-crds)
before you upgrade to a post v2.17 dex. The script mentioned in the instructions can be [found here](https://github.com/dexidp/dex/blob/v2.17.0/scripts/dump-tprs)
These instructions have been preserved for anybody who needs to use an older version of Dex and/or Kubernetes, but this is not the recommended approach. See [Migrating from TPRs to CRDs](#migrating-from-tprs-to-crds) below for information on migrating an existing installation to the new approach.
If you do wish to use TPRs, you may do so by setting the `UseTPR` flag in the storage configuration as shown below: ### Configuration
```
storage:
type: kubernetes
config:
kubeConfigFile: kubeconfig
useTPR: true
```
The `ThirdPartyResource` type acts as a description for the new resource a user wishes to create. The following an example of a resource managed by dex:
```
kind: ThirdPartyResource
apiVersion: extensions/v1beta1
metadata:
name: o-auth2-client.oidc.coreos.com
versions:
- name: v1
description: "An OAuth2 client."
```
Once the `ThirdPartyResource` is created, custom resources can be created at a namespace level (though there will be a gap between the `ThirdPartyResource` being created and the API server accepting the custom resource). While most fields are user defined, the API server still respects the common `ObjectMeta` and `TypeMeta` values. For example names are still restricted to a small set of characters, and the `resourceVersion` field can be used for an [atomic compare and swap][k8s-api].
The following is an example of a custom `OAuth2Client` resource:
```
# Standard Kubernetes resource fields
kind: OAuth2Client
apiVersion: oidc.coreos.com/v1
metadata:
namespace: foobar
name: ( opaque hash )
# Custom fields defined by dex.
clientID: "aclientid"
clientSecret: "clientsecret"
redirectURIs:
- "https://app.example.com/callback"
```
The `ThirdPartyResource` type and the custom resources can be queried, deleted, and edited like any other resource using `kubectl`.
```
kubectl get thirdpartyresources # list third party resources registered on the clusters
kubectl get --namespace=foobar oauth2clients # list oauth2 clients in a given namespace
```
To reduce administrative overhead, dex creates and manages its own third party resources and may create new ones during upgrades. While not strictly required we feel this is important for reasonable updates. Though, as a result, dex requires access to the non-namespaced `ThirdPartyResource` type. For example, clusters using RBAC authorization would need to create the following roles and bindings:
```
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: dex
rules:
- apiGroups: ["oidc.coreos.com"] # API group created by dex
resources: ["*"]
verbs: ["*"]
nonResourceURLs: []
- apiGroups: ["extensions"]
resources: ["thirdpartyresources"]
verbs: ["create"] # To manage its own resources identity must be able to create thirdpartyresources.
nonResourceURLs: []
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: dex
subjects:
- kind: ServiceAccount
name: dex # Service account assigned to the dex pod.
namespace: demo-namespace # The namespace dex is running in.
roleRef:
kind: ClusterRole
name: identity
apiVersion: rbac.authorization.k8s.io/v1alpha1
```
The storage configuration is extremely limited since installations running outside a Kubernetes cluster would likely prefer a different storage option. An example configuration for dex running inside Kubernetes: The storage configuration is extremely limited since installations running outside a Kubernetes cluster would likely prefer a different storage option. An example configuration for dex running inside Kubernetes:
@ -213,39 +137,6 @@ storage:
Dex determines the namespace it's running in by parsing the service account token automatically mounted into its pod. Dex determines the namespace it's running in by parsing the service account token automatically mounted into its pod.
## Migrating from TPRs to CRDs
This section descibes how users can migrate storage data in dex when upgrading from an older version of kubernetes (lower than 1.7). This involves creating new CRDs and moving over the data from TPRs.
The flow of the migration process is as follows:
1. Stop running old version of Dex (lower than v2.7.0).
2. Create new CRDs by running the following command:
```
kubectl apply -f scripts/manifests/crds/
```
Note that the newly created CRDs have `dex.coreos.com` as their group and will not conflict with the existing TPR resources which have `oidc.coreos.com` as the group.
3. Migrate data from existing TPRs to CRDs by running the following commands for each of the TPRs:
1. Export `DEX_NAMESPACE` to be the namespace in which the TPRs exist and run the following script to store TPR definition in a temporary yaml file:
```
export DEX_NAMESPACE="<namespace-value>"
./scripts/dump-tprs > out.yaml
```
2. Update `out.yaml` to change the apiVersion to `apiVersion: dex.coreos.com/v1` and delete the `resourceVersion` field.
```
sed 's/oidc.coreos.com/dex.coreos.com/' out.yaml
```
```
sed 's/resourceVersion: ".*"//' out.yaml
```
3. Create the resource object using the following command:
```
kubectl apply -f out.yaml
```
4. Confirm that the resource got created using the following get command:
```
kubectl get --namespace=tectonic-system <TPR-name>.dex.coreos.com -o yaml
```
4. Update to new version of Dex (v2.7.0 or higher) which will use CRDs instead of TPRs.
## SQL ## SQL
Dex supports two flavors of SQL: SQLite3 and Postgres. Dex supports two flavors of SQL: SQLite3 and Postgres.

13
scripts/dump-tprs
View File

@ -1,13 +0,0 @@
#!/bin/bash
set -e
if [ -z $DEX_NAMESPACE ];then
echo "Must export \$DEX_NAMESPACE"
exit
fi
for RESOURCE in authcodes authrequests connectors oauth2clients offlinesessionses refreshtokens passwords signingkeies; do
kubectl get --namespace=$DEX_NAMESPACE $RESOURCE.oidc.coreos.com -o yaml
done

6
storage/kubernetes/client.go
View File

@ -253,7 +253,7 @@ func (c *client) put(resource, name string, v interface{}) error {
return checkHTTPErr(resp, http.StatusOK) return checkHTTPErr(resp, http.StatusOK)
} }
func newClient(cluster k8sapi.Cluster, user k8sapi.AuthInfo, namespace string, logger log.Logger, useTPR bool) (*client, error) { func newClient(cluster k8sapi.Cluster, user k8sapi.AuthInfo, namespace string, logger log.Logger) (*client, error) {
tlsConfig := cryptopasta.DefaultTLSConfig() tlsConfig := cryptopasta.DefaultTLSConfig()
data := func(b string, file string) ([]byte, error) { data := func(b string, file string) ([]byte, error) {
if b != "" { if b != "" {
@ -329,11 +329,7 @@ func newClient(cluster k8sapi.Cluster, user k8sapi.AuthInfo, namespace string, l
} }
} }
// the API Group and version differ depending on if CRDs or TPRs are used.
apiVersion := "dex.coreos.com/v1" apiVersion := "dex.coreos.com/v1"
if useTPR {
apiVersion = "oidc.coreos.com/v1"
}
logger.Infof("kubernetes client apiVersion = %s", apiVersion) logger.Infof("kubernetes client apiVersion = %s", apiVersion)
return &client{ return &client{

26
storage/kubernetes/k8sapi/extensions.go
View File

@ -16,32 +16,6 @@ limitations under the License.
package k8sapi package k8sapi
// A ThirdPartyResource is a generic representation of a resource, it is used by add-ons and plugins to add new resource
// types to the API. It consists of one or more Versions of the api.
type ThirdPartyResource struct {
TypeMeta `json:",inline"`
// Standard object metadata
ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
// Description is the description of this object.
Description string `json:"description,omitempty" protobuf:"bytes,2,opt,name=description"`
// Versions are versions for this third party object
Versions []APIVersion `json:"versions,omitempty" protobuf:"bytes,3,rep,name=versions"`
}
// ThirdPartyResourceList is a list of ThirdPartyResources.
type ThirdPartyResourceList struct {
TypeMeta `json:",inline"`
// Standard list metadata.
ListMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
// Items is the list of ThirdPartyResources.
Items []ThirdPartyResource `json:"items" protobuf:"bytes,2,rep,name=items"`
}
// An APIVersion represents a single concrete version of an object model. // An APIVersion represents a single concrete version of an object model.
type APIVersion struct { type APIVersion struct {
// Name of this version (e.g. 'v1'). // Name of this version (e.g. 'v1').

26
storage/kubernetes/storage.go
View File

@ -38,7 +38,6 @@ const (
type Config struct { type Config struct {
InCluster bool `json:"inCluster"` InCluster bool `json:"inCluster"`
KubeConfigFile string `json:"kubeConfigFile"` KubeConfigFile string `json:"kubeConfigFile"`
UseTPR bool `json:"useTPR"` // Flag option to use TPRs instead of CRDs
} }
// Open returns a storage using Kubernetes third party resource. // Open returns a storage using Kubernetes third party resource.
@ -78,7 +77,7 @@ func (c *Config) open(logger log.Logger, waitForResources bool) (*client, error)
return nil, err return nil, err
} }
cli, err := newClient(cluster, user, namespace, logger, c.UseTPR) cli, err := newClient(cluster, user, namespace, logger)
if err != nil { if err != nil {
return nil, fmt.Errorf("create client: %v", err) return nil, fmt.Errorf("create client: %v", err)
} }
@ -86,7 +85,7 @@ func (c *Config) open(logger log.Logger, waitForResources bool) (*client, error)
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
logger.Info("creating custom Kubernetes resources") logger.Info("creating custom Kubernetes resources")
if !cli.registerCustomResources(c.UseTPR) { if !cli.registerCustomResources() {
if waitForResources { if waitForResources {
cancel() cancel()
return nil, fmt.Errorf("failed creating custom resources") return nil, fmt.Errorf("failed creating custom resources")
@ -98,7 +97,7 @@ func (c *Config) open(logger log.Logger, waitForResources bool) (*client, error)
logger.Errorf("failed creating custom resources: %v", err) logger.Errorf("failed creating custom resources: %v", err)
go func() { go func() {
for { for {
if cli.registerCustomResources(c.UseTPR) { if cli.registerCustomResources() {
return return
} }
@ -125,27 +124,17 @@ func (c *Config) open(logger log.Logger, waitForResources bool) (*client, error)
// registerCustomResources attempts to create the custom resources dex // registerCustomResources attempts to create the custom resources dex
// requires or identifies that they're already enabled. This function creates // requires or identifies that they're already enabled. This function creates
// third party resources(TPRs) or custom resource definitions(CRDs) depending // custom resource definitions(CRDs)
// on the `useTPR` flag passed in as an argument.
// It logs all errors, returning true if the resources were created successfully. // It logs all errors, returning true if the resources were created successfully.
// //
// Creating a custom resource does not mean that they'll be immediately available. // Creating a custom resource does not mean that they'll be immediately available.
func (cli *client) registerCustomResources(useTPR bool) (ok bool) { func (cli *client) registerCustomResources() (ok bool) {
ok = true ok = true
length := len(customResourceDefinitions) length := len(customResourceDefinitions)
if useTPR {
length = len(thirdPartyResources)
}
for i := 0; i < length; i++ { for i := 0; i < length; i++ {
var err error var err error
var resourceName string var resourceName string
if useTPR {
r := thirdPartyResources[i]
err = cli.postResource("extensions/v1beta1", "", "thirdpartyresources", r)
resourceName = r.ObjectMeta.Name
} else {
r := customResourceDefinitions[i] r := customResourceDefinitions[i]
var i interface{} var i interface{}
cli.logger.Infof("checking if custom resource %s has been created already...", r.ObjectMeta.Name) cli.logger.Infof("checking if custom resource %s has been created already...", r.ObjectMeta.Name)
@ -157,7 +146,6 @@ func (cli *client) registerCustomResources(useTPR bool) (ok bool) {
} }
err = cli.postResource("apiextensions.k8s.io/v1beta1", "", "customresourcedefinitions", r) err = cli.postResource("apiextensions.k8s.io/v1beta1", "", "customresourcedefinitions", r)
resourceName = r.ObjectMeta.Name resourceName = r.ObjectMeta.Name
}
if err != nil { if err != nil {
switch err { switch err {
@ -424,7 +412,7 @@ func (cli *client) DeleteRefresh(id string) error {
} }
func (cli *client) DeletePassword(email string) error { func (cli *client) DeletePassword(email string) error {
// Check for hash collition. // Check for hash collision.
p, err := cli.getPassword(email) p, err := cli.getPassword(email)
if err != nil { if err != nil {
return err return err
@ -433,7 +421,7 @@ func (cli *client) DeletePassword(email string) error {
} }
func (cli *client) DeleteOfflineSessions(userID string, connID string) error { func (cli *client) DeleteOfflineSessions(userID string, connID string) error {
// Check for hash collition. // Check for hash collision.
o, err := cli.getOfflineSessions(userID, connID) o, err := cli.getOfflineSessions(userID, connID)
if err != nil { if err != nil {
return err return err

74
storage/kubernetes/types.go
View File

@ -10,80 +10,6 @@ import (
"github.com/dexidp/dex/storage/kubernetes/k8sapi" "github.com/dexidp/dex/storage/kubernetes/k8sapi"
) )
var tprMeta = k8sapi.TypeMeta{
APIVersion: "extensions/v1beta1",
Kind: "ThirdPartyResource",
}
// The set of third party resources required by the storage. These are managed by
// the storage so it can migrate itself by creating new resources.
var thirdPartyResources = []k8sapi.ThirdPartyResource{
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "auth-code.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "A code which can be claimed for an access token.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "auth-request.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "A request for an end user to authorize a client.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "o-auth2-client.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "An OpenID Connect client.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "signing-key.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "Keys used to sign and verify OpenID Connect tokens.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "refresh-token.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "Refresh tokens for clients to continuously act on behalf of an end user.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "password.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "Passwords managed by the OIDC server.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "offline-sessions.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "User sessions with an active refresh token.",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
{
ObjectMeta: k8sapi.ObjectMeta{
Name: "connector.oidc.coreos.com",
},
TypeMeta: tprMeta,
Description: "Connectors available for login",
Versions: []k8sapi.APIVersion{{Name: "v1"}},
},
}
var crdMeta = k8sapi.TypeMeta{ var crdMeta = k8sapi.TypeMeta{
APIVersion: "apiextensions.k8s.io/v1beta1", APIVersion: "apiextensions.k8s.io/v1beta1",
Kind: "CustomResourceDefinition", Kind: "CustomResourceDefinition",