Skip to content

Azure pipeline #262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 44 commits into from
Jul 4, 2019
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
b30ac22
Azure pipeline
Jun 26, 2019
658bae8
Changes from PR review #262
Jun 27, 2019
5377dd9
Cannot provide variables without a stage definition
Jun 27, 2019
cd28a47
A world of new possibilities
Jun 27, 2019
5942dd6
Try with templates.
Jun 27, 2019
da8c55c
May work!!!
Jun 27, 2019
a5f381c
Test
Jun 27, 2019
4d98086
Rename the template file.
Jun 27, 2019
2fd38ba
Another test for displayNames to be pretty
Jun 27, 2019
a17e3da
Forgot some quotes.
Jun 27, 2019
54534d3
I may be loosing my mind slowly
Jun 27, 2019
9bb3535
Another test
Jun 27, 2019
bd08e8f
Last ty
Jun 27, 2019
a639ff5
Should work but not super pretty
Jun 27, 2019
0cdfd74
Instead of running step on condition, do not add it on condition.
Jun 27, 2019
f35d489
forgot a $
Jun 27, 2019
d0170c0
revert the changes which were not working, but kept some docs.
Jun 27, 2019
322abe9
Triggered only for PRs, not every branch for every commit.
Jun 28, 2019
4c13fb4
Remove publishing steps from the `whole build`. Instead, copy the cre…
Jun 28, 2019
9f16bf4
Publish all the created artifacts if succeeded.
Jun 28, 2019
7f5dd9f
Trigger on tags starting with the letter v
Jun 28, 2019
625a973
Why is it building master???
Jun 28, 2019
68a3614
Is it the PR?
Jun 28, 2019
e6eebd9
Stage to have the publishing one depending on the building one.
Jun 28, 2019
e11c639
Need to publish artifacts per job.
Jun 28, 2019
6ddecc8
Update back the publish artifact step
Jun 28, 2019
65651f0
Let's try the release pipelines.
Jun 28, 2019
f477e86
Add some documentation.
Jun 28, 2019
f511546
ignore folders `vnv`.
Jun 28, 2019
b8d7e99
Publish test results even if tests failed.
Jul 1, 2019
f47fe0c
Build every tag
Jul 1, 2019
687a107
Version v0.13.1-b2
Jul 1, 2019
24cd5ec
Version 0.13.1-b3
Jul 1, 2019
f91870e
Forgot a star
Jul 1, 2019
fc34b07
Was the space really important? And version beta up
Jul 1, 2019
ddcb2b2
Build is `Automated` for every tag, not only the one starting with `v`.
Jul 1, 2019
15ab9f2
Add a MANIFEST.in file to include the read me and the license files i…
Jul 1, 2019
e0e04b2
Bump beta version for final test
Jul 1, 2019
7ac1925
Update some docs.
Jul 1, 2019
a14e8a0
MANIFEST is not the issue.
Jul 1, 2019
f7241df
It seems building a wheel from a windows machine is complicated with …
Jul 1, 2019
5c3adb9
The published packaged version seems to include the readme.
Jul 1, 2019
7ba0bc2
Should be the final version bump for test.
Jul 1, 2019
0c97b1c
Merge branch 'master' of github.com:data61/clkhash into feature-azure…
Jul 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .azurePipeline/wholeBuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
parameters:
pythonVersions: []
operatingSystems: []
architectures: []

jobs:
- job: 'test_and_publish'
strategy:
matrix:
${{ each py in parameters.pythonVersions }}:
${{ each os in parameters.operatingSystems }}:
${{ each architecture in parameters.architectures }}:
${{ format('{0}_Python_{1}_{2}', os, py, architecture) }}:
pythonVersion: ${{ py }}
operatingSystem: ${{ os }}
architecture: ${{ architecture }}

pool:
vmImage: $(operatingSystem)

displayName: 'Test and publish for '

steps:
# I would not have found on my own such a step. Thanks https://dvlup.com/2019/01/03/using-powershell-to-installing-msi-in-a-devops-build/
# The solution I found was not working: it hanged forevever, using "msiexec /i"
# These steps are required because of at least the bitarray dependency (but maybe more). See issue #153 tracking if bitarray is still used.
- powershell: Invoke-WebRequest -Uri https://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi -OutFile VCForPython27.msi
displayName: 'Download Microsoft Visual C++ 9.0 from https://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi'
condition: eq(variables['pythonVersion'], 2.7)
- powershell: Start-Process VCForPython27.msi -ArgumentList "/q" -Wait
displayName: 'Install Microsoft Visual C++ 9.0.'
condition: and(eq(variables['operatingSystem'], 'vs2017-win2016'), eq(variables['pythonVersion'], 2.7))

- task: UsePythonVersion@0
inputs:
versionSpec: '$(pythonVersion)'
architecture: '$(architecture)'

- script: |
python -m pip install -U pip
python -m pip install -U wheel setuptools codecov
python -m pip install -U -r requirements.txt
python -m pip install .
displayName: 'Install requirements'

- script: python -m pytest --cov=clkhash --junitxml=testResults.xml --cov-report=xml:coverageReport.xml --cov-report=html:htmlcov
displayName: 'Test with pytest'
env:
INCLUDE_CLI: 1

- task: PublishTestResults@2
condition: succeeded()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean it won't publish test results if they don't all pass?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well seen :)

displayName: 'Publish test results'
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'testResults.xml'
testRunTitle: 'Test results on a vm $(operatingSystem) ($(architecture)) for Python $(pythonVersion)'
failTaskOnFailedTests: true

- task: PublishCodeCoverageResults@1
# If the previous stage fail, we still want to run this one as the previous stage may fail because of a failing test.
condition: succeededOrFailed()
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: 'coverageReport.xml'
# Seems to create warning as this step is already creating its own html pages.
# reportDirectory: 'htmlcov'
failIfCoverageEmpty: true

- script: python -m codecov --token $(CODECOV_TOKEN) --file coverageReport.xml
displayName: 'Send coverage to codecov'

- script: |
pyinstaller cli.spec
.\dist\clkutil.exe --version
displayName: 'Build clkutil.exe'
condition: and(eq(variables['operatingSystem'], 'vs2017-win2016'), eq(variables['pythonVersion'], '3.7'), eq(variables['architecture'], 'x86'))

- script: python setup.py sdist bdist_wheel
displayName: 'Package'
condition: and(eq(variables['pythonVersion'], '3.7'), eq(variables['architecture'], 'x86'))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we package for every combo, and move this up (after installing requirements). we can remove line 43 (python -m pip install .) and instead test the packaged version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed, but could you check that it is doing what you would like? I didn't provide to pytest the dist, and it's still working.


- script: |
pip install twine
twine upload -u $(PYPI_LOGIN) -p $(PYPI_PASSWORD) --skip-existing dist/*.whl
displayName: 'Send artifact to Pypi'
condition: and(eq(variables['pythonVersion'], '3.7'), eq(variables['architecture'], 'x86'))
Copy link
Collaborator

@hardbyte hardbyte Jun 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should only publish to pypi on tagged commit, or for now perhaps only when manually chosen?

Now that you're publishing the pipeline artifact can this easily be moved into own job?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue with tagged commits is that the tagging is usually happening after the commit have been pushed. So to still do that and have a manual confirmation, I had to create a release pipeline which should be triggered after a tagged version is built from master.


- task: PublishPipelineArtifact@0
inputs:
artifactName: "$(operatingSystem) artifacts"
targetPath: dist/
condition: and(eq(variables['pythonVersion'], '3.7'), eq(variables['architecture'], 'x86'))
8 changes: 8 additions & 0 deletions azurePipeline.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
jobs:
- template: .azurePipeline/wholeBuild.yml # Template reference
parameters:
pythonVersions: ['3.7', '3.6', '3.5', '2.7']
# operatingSystems: ['ubuntu-16.04', 'macos-10.13', 'vs2017-win2016']
operatingSystems: ['vs2017-win2016']
architectures: ['x64', 'x86']