Move all environment variable logic to config data?

[mattldawson]

Several components (e.g., endpoints, databases) require information that is expected to be present in environment variables (ids, API secrets, etc.). In some cases, the environment variables are embedded in the yaml config data and extracted via the config package, e.g., Globus endpoints:

dts/endpoints/globus/endpoint.go

Lines 118 to 135 in cc47140

	func NewEndpointFromConfig(endpointName string) (endpoints.Endpoint, error) {
	epConfig, found := config.Endpoints[endpointName]
	if !found {
	return nil, fmt.Errorf("'%s' is not an endpoint", endpointName)
	}
	if epConfig.Provider != "globus" {
	return nil, fmt.Errorf("'%s' is not a Globus endpoint", endpointName)
	}
	credential, found := config.Credentials[epConfig.Credential]
	if !found {
	return nil, fmt.Errorf("invalid credential for endpoint '%s': %s", endpointName, epConfig.Credential)
	}
	clientId, err := uuid.Parse(credential.Id)
	if err != nil {
	return nil, fmt.Errorf("invalid Globus client ID for credential '%s': %s (must be UUID)", epConfig.Credential, credential.Id)
	}
	return NewEndpoint(epConfig.Name, epConfig.Id, epConfig.Root, clientId, credential.Secret)
	}

However, in other cases, functions that create new components look for the environment variables directly, e.g., JDP database:

dts/databases/jdp/database.go

Lines 67 to 72 in cc47140

	func NewDatabase() (databases.Database, error) {
	// make sure we have a shared secret or an SSO token
	secret, haveSecret := os.LookupEnv("DTS_JDP_SECRET")
	if !haveSecret { // check for SSO token
	return nil, fmt.Errorf("no shared secret was found for JDP authentication")
	}

Would it make sense to have a single approach to handling this data? My recommendation would be to include the environment variables in the yaml config data. This would make testing with mocked databases/endpoints more straightforward.

[jeff-cohere]

I think this has been completed. Is that right @mattldawson ?