Feature(command-dev): --toml <path-to-toml>

[bdefore]

Is your feature request related to a problem? Please describe.

Using environment variables in netlify.toml is not supported. Firstly, I find this to be a prominent shortcoming of the Netlify service, and if this were supported, this feature request would be unnecessary.

Some of my site's settings here are either secrets that I don't want in version control or benefit from dynamic assignments, so I use a workaround similar to the last option in https://docs.netlify.com/configure-builds/file-based-configuration/#inject-environment-variable-values

I began with sed replacement as Netlify suggested, but soon found it error prone and unwieldy, so I made a Node script: https://gist.github.com/bdefore/29a40a1028cf414fe51c1b39686765bc (in this file I populate the contents of required and optional with variables relevant to my site).

My configuration for command in netlify.toml is: CI= yarn run build:replace && CI= yarn run build and this works well during deployments, replacing placeholders with those specified in site configuration.

Where this becomes problematic is when I want to run Netlify Dev, which would rely on a replaced version of netlify.toml. I currently must run the rewrite script locally once before starting Netlify Dev: "build:replace:dev": "netlify dev:exec babel-node rewrite_netlify_vars.js", in particular because I rely on netlify.toml for [redirects] blocks even in local development.

This replaces placeholder values in netlify.toml but since the file is managed by git, there is a risk of accidentally committing the changes and leaking secrets into version control.

Describe the solution you'd like

I'd like to have a .gitignore'd netlify.development.toml which is the output of a replace run, and be able to run netlify dev --toml netlify.development.toml.

Alternatively, the [dev] block could support a localOverrides that is a path to another toml that is internally merged before Netlify Dev runs.

Describe alternatives you've considered

• Look for relevant Netlify plugins
• Remain careful and don't commit replaced netlify.toml
• Write a script that watches Netlify Dev terminating and restores the file.
• Work off of a long-living local branch that is regularly rebased with master and is blocked from being pushed

Additional context

Related discussion that proposes ability to override toml values individually: #1265
Related discussion about supporting environment variables in netlify.toml: #457

Can you submit a pull request?

Possibly

Pull requests are welcome! If you would like to help us add this feature, please check our
contributions guidelines.

[bdefore]

EDIT: note that I slightly modified the described. Continued in #1599 (comment)

I was able to find a reasonable workaround here, since I found that my Create React App based application server could own proxying to the deployed production where redirects were then configured to work as needed: https://create-react-app.dev/docs/proxying-api-requests-in-development/

Changes I made from proposed above:

• Removed [dev] block from netlify.toml and placed it in a netlify.development.toml
• Changed my build:replace:dev to instead do cp netlify.development.toml netlify.toml

This means that if I were to inadvertently commit netlify.toml with the contents of netlify.development.toml, no secrets would enter version control. If I inadvertently pushed, Netlify would fail to find a [build] block to do anything with and fail loudly. (This presumes no settings specified through the dashboard site configuration)

It does mean that I have an added inconvenience of having to git checkout netlify.toml for every commit, but it is at least safe if I forget to do so. I could alternatively add netlify.toml to my .gitignore and force override it when intending to make a change.

Based on what I see here #640 it sounds like Netlify Dev does not intend to process [[redirects]] so perhaps I have encountered a regression.

Here is my netlify.development.toml:

[dev]
  command = "yarn start:dev" # Command to start your dev server
  functions = "private/lambda"
  targetPort = 8080 # The port for your application server, framework or site generator
  # port = 8888 # The port that the netlify dev will be accessible on
  publish = "build/" # The path to your static content folder
  autoLaunch = false # a Boolean value that determines if Netlify Dev launches the local server address in your browser
  # https://github.com/netlify/cli/issues/1300
  framework = "#custom" # https://cli.netlify.com/netlify-dev/#project-detection

[erezrokah]

Hi @bdefore, thank you for the detailed description. Can you outline the specific netlify.toml settings you're replacing?
If those are just environment variables, you could use a .env file and netlify dev will read values from there.

[AlexKasaku]

I believe @bdefore is using token replacements within [[Redirects]], similar to us:

[[redirects]]
	from = '/api/*'
	to = '{API_HOST}/:splat'
	status = 200
	force = true
	[redirects.headers]
    	Authorization = "{API_HOST_AUTH_HEADER}"

Since Netlify does not support using environment variables here, the suggested approach is to preprocess the netlify.toml file as part of the build, which is what we do, see here: https://docs.netlify.com/configure-builds/file-based-configuration/#inject-environment-variable-values

We would like to maintain the netlify.toml file, so being able to pass in a reference to an already-processed netlify.dev.toml file to netlify dev would be useful for this use case.

[letoast]

+1 - having a different workflow for build and dev is not desirable. Other services already have the option of referencing outside environment variables in their build configuration files.

[bdefore]

I still use the above workaround, but inverted. A base configuration needed for Netlify Dev constitutes my netlify.toml. When running in Netlify's CI my first step is to append supplemental configuration required for a deployed environment, such as redirects. These live in a netlify.ci.toml and I just cat netlify.ci.toml >> netlify.toml as part of command. I changed to this way primarily because netlify.toml is committed to source and I didn't want to inadvertently ship the dev configuration.