Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github actions updates #139

Merged
merged 56 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
d77f7de
Update build-codeql.yaml test codeql action
jacob-ronstadt Apr 29, 2024
809d01d
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
a2502d0
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
efbbbb4
Update build-codeql.yaml manual download
jacob-ronstadt Apr 29, 2024
e70117f
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
23cd95f
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
982a00a
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
13c00c5
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
e091a07
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
db0da1f
Update build-codeql.yaml add back yest of build step
jacob-ronstadt Apr 29, 2024
86f227c
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
43105fd
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
2f467e9
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
34dbfa5
Update build-codeql.yaml try all jobs
jacob-ronstadt Apr 29, 2024
e86615d
Update build-codeql.yaml test all jobs
jacob-ronstadt Apr 29, 2024
ffc595b
Update build-codeql.yaml dvl tests
jacob-ronstadt Apr 29, 2024
0093aa5
Add files via upload
jacob-ronstadt Apr 29, 2024
7c4a30a
Create readme
jacob-ronstadt Apr 29, 2024
d6246bd
restructure
jacob-ronstadt Apr 29, 2024
ccfd673
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
b96dbd7
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
e9759f3
add test projects
jacob-ronstadt Apr 29, 2024
1987efb
Update dvl_tests.ps1
jacob-ronstadt Apr 29, 2024
71b6e99
Update dvl_tests.ps1
jacob-ronstadt Apr 29, 2024
fc5eb22
Update build-codeql.yaml
jacob-ronstadt Apr 29, 2024
3d8f149
Update dvl_tests.ps1
jacob-ronstadt Apr 29, 2024
5b9e202
Update dvl_tests.ps1
jacob-ronstadt Apr 29, 2024
0e16a3e
Update dvl_tests.ps1
jacob-ronstadt Apr 29, 2024
f9402c2
update build_create_analyze_test.py to fail when different results ar…
jacob-ronstadt Apr 30, 2024
91efca0
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
4393abc
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
80e08ca
change when results get uploaded
jacob-ronstadt Apr 30, 2024
cc26e80
Merge branch 'jacob-ronstadt/github_actions_update' of https://github…
jacob-ronstadt Apr 30, 2024
3be2b60
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
dc30d17
fix exit code
jacob-ronstadt Apr 30, 2024
cff0917
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
0557a1e
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
a8f2cf1
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
28cf0ca
Update build-codeql.yaml
jacob-ronstadt Apr 30, 2024
4916dfa
update to use driver_snippet.c instead of other driver projects
jacob-ronstadt May 13, 2024
a9c8981
Merge branch 'jacob-ronstadt/github_actions_update' of https://github…
jacob-ronstadt May 13, 2024
b04a9fb
add template path variable
jacob-ronstadt May 13, 2024
79b00ae
fix path
jacob-ronstadt May 13, 2024
bee078c
fix path
jacob-ronstadt May 13, 2024
f1ecf24
test
jacob-ronstadt May 13, 2024
e3dba3c
fix template path
jacob-ronstadt May 13, 2024
8ed9f1f
test
jacob-ronstadt May 13, 2024
5a688e5
update template project to have arm configuration
jacob-ronstadt May 13, 2024
d53b4c3
fix msbuild command
jacob-ronstadt May 13, 2024
87cf468
fix driver_snippet.c
jacob-ronstadt May 13, 2024
5b78371
Update build-codeql.yaml
jacob-ronstadt May 13, 2024
b88bdc2
print results location
jacob-ronstadt May 15, 2024
7bd8a2c
Update build-codeql.yaml
jacob-ronstadt May 15, 2024
dc8f753
fix sarif copy and compare sarif results
jacob-ronstadt May 15, 2024
ac880ee
add expected sarif results
jacob-ronstadt May 15, 2024
39d7bba
Delete src/drivers/test/dvl_tests/readme
jacob-ronstadt May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 188 additions & 34 deletions .github/workflows/build-codeql.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,11 @@ on:

# Allow manual scheduling
workflow_dispatch:

jobs:
build-publish:
build:
runs-on: windows-latest

permissions:
contents: read
packages: write
Expand All @@ -30,22 +31,58 @@ jobs:
path: .
fetch-depth: 0

- name: Download CodeQL CLI
uses: i3h/[email protected]
with:
owner: "github"
repo: "codeql-cli-binaries"
tag: "v2.15.4"
file: "codeql-win64.zip"
- name: CodeQL Download
run:
Invoke-WebRequest -Uri "https://github.com/github/codeql-cli-binaries/releases/download/v${{ vars.CODEQL_VERSION }}/codeql-win64.zip" -OutFile codeql-win64.zip;
Expand-Archive -Path codeql-win64.zip -DestinationPath .\codeql-zip -Force;
Move-Item -Path .\codeql-zip\codeql -Destination .\codeql-cli\

- name: Install CodeQL pack dependencies
shell: cmd
run: |
pushd .\src
..\codeql-cli\codeql.cmd pack install
popd

- name: codeql version test
run: .\codeql-cli\codeql.exe version

- name: Build must-fix driver suite
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only windows_mustfix_partial.qls

- name: Unzip CodeQL CLI
run: Expand-Archive -Path codeql-win64.zip -DestinationPath .\codeql-zip -Force
- name: Build recommended driver suite
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only windows_recommended_partial.qls

- name: Move CodeQL CLI folder to main subdirectory
- name: Build CA ported queries
shell: cmd
continue-on-error: true # Required because robocopy returns 1 on success
run: robocopy /S /move .\codeql-zip\codeql .\codeql-cli\

run: .\codeql-cli\codeql.cmd query compile --check-only ported_driver_ca_checks.qls

- name: Build all Windows queries
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only .\src

test-query-health:
runs-on: windows-latest
needs: build
permissions:
contents: read
packages: write
steps:
- name: Enable long git paths
shell: cmd
run: git config --global core.longpaths true
- name: Clone self (windows-driver-developer-supplemental-tools)
uses: actions/checkout@v4
with:
path: .
fetch-depth: 0
- name: CodeQL Download
run:
Invoke-WebRequest -Uri "https://github.com/github/codeql-cli-binaries/releases/download/v${{ vars.CODEQL_VERSION }}/codeql-win64.zip" -OutFile codeql-win64.zip;
Expand-Archive -Path codeql-win64.zip -DestinationPath .\codeql-zip -Force;
Move-Item -Path .\codeql-zip\codeql -Destination .\codeql-cli\
- name: Install CodeQL pack dependencies
shell: cmd
run: |
Expand All @@ -54,47 +91,94 @@ jobs:
popd
- name: codeql version test
run: .\codeql-cli\codeql.exe version

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.11

- name: Install Python Packages
run: |
python -m pip install --upgrade pip
pip install -r .\src\drivers\test\requirements.txt
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2

- name: Run test script
shell: pwsh
continue-on-error: true # Allow script to return non-zero exit code
env:
CONNECTION_STRING: ${{ secrets.CONNECTION_STRING }}
ACCOUNT_KEY: ${{ secrets.ACCOUNT_KEY }}
SHARE_NAME: ${{ secrets.SHARE_NAME }}
CONTAINER_NAME: ${{ secrets.CONTAINER_NAME }}
ACCOUNT_NAME: ${{ secrets.ACCOUNT_NAME }}

run: python src\drivers\test\build_create_analyze_test.py --codeql_path .\codeql-cli\codeql.exe --no_build --compare_results --connection_string "$env:CONNECTION_STRING" --share_name "$env:SHARE_NAME" --container_name "$env:CONTAINER_NAME" --storage_account_key "$env:ACCOUNT_KEY" --storage_account_name "$env:ACCOUNT_NAME"
run: python src\drivers\test\build_create_analyze_test.py --codeql_path .\codeql-cli\codeql.exe --no_build --compare_results --connection_string "$env:CONNECTION_STRING" --share_name "$env:SHARE_NAME" --container_name "$env:CONTAINER_NAME"

- name: Build must-fix driver suite
test-codeql-latest-vs-current:
# Tests if the latest codeql version produces the same results as the current version.
runs-on: windows-latest
continue-on-error: true # Allow script to return non-zero exit code
needs: [build,test-query-health]
permissions:
contents: read
packages: write
steps:
- name: Enable long git paths
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only windows_mustfix_partial.qls
run: git config --global core.longpaths true

- name: Build recommended driver suite
- name: Clone self (windows-driver-developer-supplemental-tools)
uses: actions/checkout@v4
with:
path: .
fetch-depth: 0
- name: CodeQL Download
run:
Invoke-WebRequest -Uri "https://github.com/github/codeql-cli-binaries/releases/download/v${{ vars.CODEQL_LATEST_VERSION }}/codeql-win64.zip" -OutFile codeql-win64.zip;
Expand-Archive -Path codeql-win64.zip -DestinationPath .\codeql-zip -Force;
Move-Item -Path .\codeql-zip\codeql -Destination .\codeql-cli\
- name: Install CodeQL pack dependencies
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only windows_recommended_partial.qls

- name: Build CA ported queries
run: |
pushd .\src
..\codeql-cli\codeql.cmd pack install
popd
- name: codeql version test
run: .\codeql-cli\codeql.exe version
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.11
- name: Install Python Packages
run: |
python -m pip install --upgrade pip
pip install -r .\src\drivers\test\requirements.txt
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2
- name: Run test script
shell: pwsh
env:
CONNECTION_STRING: ${{ secrets.CONNECTION_STRING }}
ACCOUNT_KEY: ${{ secrets.ACCOUNT_KEY }}
SHARE_NAME: ${{ secrets.SHARE_NAME }}
CONTAINER_NAME: ${{ secrets.CONTAINER_NAME }}
ACCOUNT_NAME: ${{ secrets.ACCOUNT_NAME }}
run: python src\drivers\test\build_create_analyze_test.py --codeql_path .\codeql-cli\codeql.exe --no_build --compare_results --connection_string "$env:CONNECTION_STRING" --share_name "$env:SHARE_NAME"

test-pack-version-update:
runs-on: windows-latest
needs: build
permissions:
contents: read
packages: write
steps:
- name: Enable long git paths
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only ported_driver_ca_checks.qls
run: git config --global core.longpaths true

- name: Build all Windows queries
shell: cmd
run: .\codeql-cli\codeql.cmd query compile --check-only .\src

- name: Clone self (windows-driver-developer-supplemental-tools)
uses: actions/checkout@v4
with:
path: .
fetch-depth: 0

- name: Check for changes to qlpack
shell: pwsh
run:
Expand All @@ -118,16 +202,86 @@ jobs:
try{$old_qlpack_version = [version]($qlpack_changes -match "-version").Substring(10);} catch {"Changed qlpack.yml without updating version"; exit 1 }
try{$new_qlpack_version = [version]($qlpack_changes -match "\+version").Substring(10);} catch {"Changed qlpack.yml without updating version"; exit 1 }
if ($new_qlpack_version -gt $old_qlpack_version) { exit 0 } else { "qlpack.yml version not incremented"; exit 1 }

test-create-dvl:
runs-on: windows-latest
needs: build
permissions:
contents: read
packages: write
steps:
- name: Enable long git paths
shell: cmd
run: git config --global core.longpaths true

- name: Clone self (windows-driver-developer-supplemental-tools)
uses: actions/checkout@v4
with:
path: .
fetch-depth: 0

- name: CodeQL Download
run:
Invoke-WebRequest -Uri "https://github.com/github/codeql-cli-binaries/releases/download/v${{ vars.CODEQL_VERSION }}/codeql-win64.zip" -OutFile codeql-win64.zip;
Expand-Archive -Path codeql-win64.zip -DestinationPath .\codeql-zip -Force;
Move-Item -Path .\codeql-zip\codeql -Destination .\codeql-cli\

- name: Install CodeQL pack dependencies
shell: cmd
run: |
pushd .\src
..\codeql-cli\codeql.cmd pack install
popd
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2

- name: Test DVL
run: src\drivers\test\dvl_tests\dvl_tests.ps1

- name: Archive code coverage results
uses: actions/upload-artifact@v4
with:
name: dvl-outputs
path: |
clean_results\*.*
mustfix_results\*.*

publish:
runs-on: windows-latest
needs: [build, test-pack-version-update, test-query-health]
permissions:
contents: read
packages: write
steps:
- name: Enable long git paths
shell: cmd
run: git config --global core.longpaths true

- name: Clone self (windows-driver-developer-supplemental-tools)
uses: actions/checkout@v4
with:
path: .
fetch-depth: 0

- name: CodeQL Download
run:
Invoke-WebRequest -Uri "https://github.com/github/codeql-cli-binaries/releases/download/v${{ vars.CODEQL_VERSION }}/codeql-win64.zip" -OutFile codeql-win64.zip;
Expand-Archive -Path codeql-win64.zip -DestinationPath .\codeql-zip -Force;
Move-Item -Path .\codeql-zip\codeql -Destination .\codeql-cli\

- name: Install CodeQL pack dependencies
shell: cmd
run: |
pushd .\src
..\codeql-cli\codeql.cmd pack install
popd
- name: Publish New CodeQL Pack
shell: pwsh
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

run:
$build = git rev-parse --short HEAD;
$version =( Select-String .\src\qlpack.yml -Pattern "version").line;
$new_ver = "$version-alpha+$build";
(Get-Content .\src\qlpack.yml).Replace($version, $new_ver) | Set-Content .\src\qlpack.yml;
.\codeql-cli\codeql.cmd pack publish --allow-prerelease ./src;

15 changes: 11 additions & 4 deletions src/drivers/test/WDMTestTemplate/WDMTestTemplate.sln
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,25 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "fail_driver1", "driver\fail
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|ARM64 = Debug|ARM64
Debug|Win32 = Debug|Win32
Debug|x64 = Debug|x64
Release|ARM64 = Release|ARM64
Release|Win32 = Release|Win32
Release|x64 = Release|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|Win32.ActiveCfg = Debug|Win32
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|Win32.Build.0 = Debug|Win32
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|ARM64.ActiveCfg = Debug|ARM64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|ARM64.Build.0 = Debug|ARM64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|ARM64.Deploy.0 = Debug|ARM64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|Win32.ActiveCfg = Debug|x64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|Win32.Build.0 = Debug|x64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|x64.ActiveCfg = Debug|x64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Debug|x64.Build.0 = Debug|x64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|Win32.ActiveCfg = Release|Win32
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|Win32.Build.0 = Release|Win32
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|ARM64.ActiveCfg = Release|ARM64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|ARM64.Build.0 = Release|ARM64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|ARM64.Deploy.0 = Release|ARM64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|Win32.ActiveCfg = Release|x64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|x64.ActiveCfg = Release|x64
{7F88D5C5-F05F-4817-89F5-C811053277A0}.Release|x64.Build.0 = Release|x64
EndGlobalSection
Expand Down
Loading