Add a pre-process method

[mikecooke77]

Description

This PR addresses the second part of #25 by adding a new method which can be called using yp-preprocessor.

Issue(s) addressed

Resolves #25

Dependencies

No dependencies.

Impact

None

Checklist

• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have run the unit tests before creating the PR

[DavidSimonin]

I made few comments, but I wuld be happy eitherway for most of them.

[DavidSimonin]

Happy with your comment above.

Regarding the the enviroment variable, should we use something like that to evalusate them instead of a mapping?:

>>> os.path.expandvars('$DATADIR/test/testing')
'/data/users/frsd/test/testing'

[mikecooke77]

os.path.expandvars('$DATADIR/test/testing')

I like this as an idea but we are not passing them in as environment variables but as a defined list so I can't see how to get this to work. Unless we want to define it as an environment variable which I would prefer not to do because the define method makes the user do this explicitly.

[matthewrmshin]

See also yp-data's help on how environment variables are mapped.

• Environment variables are the default mapping for variable syntax in the file, unless the --no-environment option is specified.
• Each -D KEY=VALUE overrides.
• There is also a --no-process-variable option.

[mikecooke77]

Thanks for the comments I have looked at the code on the dataprocessor and used this to update the datapreprocessor so that environment variables can also be used (and turned off).

I don't think for the preprocessor we would ever need to no process variables.

[mikecooke77]

@matthewrmshin if you are happy with this code could we get it in the stack I can them start testing the jjdoc / jjaux split which will enable us to get up to date