Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The active test run was aborted. Reason: Test host process crashed #451

Open
3 tasks
fanyang-mono opened this issue Jul 25, 2023 · 8 comments
Open
3 tasks

Comments

@fanyang-mono
Copy link
Member

fanyang-mono commented Jul 25, 2023

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=351450

Build leg reported

chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution

Pull Request

dotnet/runtime#89217

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The active test run was aborted. Reason: Test host process crashed",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
912714 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76695
2616802 dotnet-interactive Test / Blame Log
2616717 dotnet-interactive Test / Blame Log
2615694 dotnet-fsharp CmdLine@2 Log
2615469 dotnet-interactive Test / Blame Log
2614768 dotnet-interactive Test / Blame Log
909122 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75970
2614588 dotnet-fsharp CmdLine@2 Log
908618 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76649
908555 dotnet/roslyn Run Integration Tests Log
908425 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76646
2614116 dotnet-interactive Test / Blame Log
2614101 dotnet-interactive Test / Blame Log
2613990 dotnet-interactive Test / Blame Log
2613544 dotnet-interactive Test / Blame Log
2612493 dotnet-interactive Test / Blame Log
906308 dotnet/roslyn Run Integration Tests Log
2612433 dotnet-interactive Test / Blame Log
2612333 dotnet-interactive Test / Blame Log
905800 dotnet/roslyn Run Integration Tests Log
2611980 dotnet-interactive Test / Blame Log
2611872 dotnet-interactive Test / Blame Log
2611807 dotnet-interactive Test / Blame Log
905108 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#70455
905102 dotnet/roslyn Run Integration Tests Log
903012 dotnet/roslyn Run Integration Tests Log
900073 dotnet/roslyn Run Integration Tests Log
899709 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76517
899492 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75581
899447 dotnet/roslyn Run Integration Tests Log
899394 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76509
899197 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76509
899158 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76504
899075 dotnet/roslyn Run Integration Tests Log
898930 dotnet/roslyn Run Integration Tests Log
898763 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76490
898150 dotnet/roslyn Run Integration Tests Log
898015 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76479
897448 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76437
897339 dotnet/roslyn Run Integration Tests Log
897311 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76459
897138 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76466
895268 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76427
895169 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76424
895154 dotnet/roslyn Run Integration Tests Log
895032 dotnet/roslyn Run Integration Tests Log
894592 dotnet/roslyn Run Integration Tests Log
894433 dotnet/roslyn Run Integration Tests Log
894400 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76411
894342 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76413
894314 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76386
894260 dotnet/roslyn Run Integration Tests Log
894249 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76408
894156 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76351
894113 dotnet/roslyn Run Integration Tests Log
893996 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76358
893920 dotnet/roslyn Run Integration Tests Log
893668 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76235
893200 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76387
893046 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76384
892987 dotnet/roslyn Run Integration Tests Log
892776 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76357
892595 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76375
Build Definition Test Pull Request
913226 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45852
913232 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45848
913087 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45858
912947 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45852
912665 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45844
912738 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureServiceBusExtensionsTests.Aspire.Hosting.Azure.Tests.AzureServiceBusExtensionsTests.VerifyWaitForOnServiceBusEmulatorBlocksDependentResources
912469 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45812
912716 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45813
912694 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureServiceBusExtensionsTests.Aspire.Hosting.Azure.Tests.AzureServiceBusExtensionsTests.VerifyWaitForOnServiceBusEmulatorBlocksDependentResources dotnet/aspire#7014
912701 dotnet/aspire Aspire.Hosting.Azure.Tests.WorkItemExecution
912430 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45806
912410 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45819
912436 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#44268
912350 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45795
912267 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45801
912185 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7048
912243 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#44268
912222 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45830
912199 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution dotnet/sdk#45815
911662 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45793
911973 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45724
912106 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45804
912072 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45823
912013 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45804
912027 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45633
911967 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45830
911958 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45829
911962 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45827
911790 dotnet/sdk Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution dotnet/sdk#45538
2616449 dotnet-roslyn Microsoft.CodeAnalysis.VisualBasic.Features.UnitTests_240.WorkItemExecution
911475 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45813
911640 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45823
911491 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45816
911539 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45819
911476 dotnet/sdk Microsoft.NET.ToolPack.Tests.dll.3.WorkItemExecution dotnet/sdk#45812
911245 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45808
911253 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45749
911158 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45804
910003 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45777
911061 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#45796
911155 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.ElasticsearchFunctionalTests.Aspire.Hosting.Elasticsearch.Tests.ElasticsearchFunctionalTests.VerifyWaitForOnElasticsearchBlocksDependentResources dotnet/aspire#7014
911038 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45550
911104 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45797
910900 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#45782
910845 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.1.WorkItemExecution dotnet/sdk#45794
910771 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureServiceBusExtensionsTests.Aspire.Hosting.Azure.Tests.AzureServiceBusExtensionsTests.VerifyWaitForOnServiceBusEmulatorBlocksDependentResources dotnet/aspire#7005
910832 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45749
910622 dotnet/sdk Microsoft.DotNet.Watch.UnitTests.ApplyDeltaTests.Microsoft.DotNet.Watch.UnitTests.ApplyDeltaTests.ProjectChangeAndRestart dotnet/sdk#45603
910710 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45792
910683 dotnet/aspire Aspire.Hosting.Redis.Tests.RedisFunctionalTests.Aspire.Hosting.Redis.Tests.RedisFunctionalTests.RedisInsightWithDataShouldPersistStateBetweenUsages(useVolume: False) dotnet/aspire#7045
910468 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45786
910459 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#44523
910508 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45724
910009 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45780
910011 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45709
909298 dotnet/sdk Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution dotnet/sdk#45754
909742 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
909500 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45725
909693 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45767
909667 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45763
909509 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#44523
909595 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution dotnet/sdk#45766
908912 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.19.WorkItemExecution dotnet/sdk#45598
909217 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45749
909183 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45724
908876 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45745
909020 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45748
908879 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45720
908710 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.21.WorkItemExecution dotnet/sdk#45732
908777 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45744
908713 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45731
908614 dotnet/aspnetcore InMemory.FunctionalTests--net10.0.WorkItemExecution
908198 dotnet/aspire Aspire.Hosting.Redis.Tests.WorkItemExecution dotnet/aspire#6946
908266 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution dotnet/sdk#45383
908165 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution dotnet/sdk#45383
907950 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45732
907925 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#45728
907726 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.20.WorkItemExecution dotnet/sdk#45726
907619 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#45418
907391 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45354
907492 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45618
907365 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45720
907324 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45722
907328 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45720
907326 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45723
907310 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45721
907233 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45718
907205 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45717
906666 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45712
906836 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45703
906525 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45706
906969 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.1.WorkItemExecution dotnet/sdk#45702
906825 dotnet/sdk Microsoft.NET.Build.Tests.dll.9.WorkItemExecution dotnet/sdk#45710
906813 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45709
906814 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45708
906758 dotnet/aspire Aspire.Hosting.Kafka.Tests.WorkItemExecution dotnet/aspire#7021
906731 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45705
906665 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45713
906556 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45705
906526 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45707
Displaying 100 of 295 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
27 123 358

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 2/8/2024 9:24:30 PM UTC

@andriipatsula
Copy link
Member

Hello @fanyang-mono, could you please update the "ErrorMessage" : "" by following the step by step documentation on how to create a known issue

@fanyang-mono
Copy link
Member Author

Updated.

@pavelsavara
Copy link
Member

pavelsavara commented Nov 7, 2023

This is duplicate of dotnet/runtime#89409, except here it's not only wasm debugger

@dougbu
Copy link
Member

dougbu commented Nov 23, 2023

@pavelsavara and @fanyang-mono, should this issue be closed in favour of dotnet/runtime#94560❔ if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

@pavelsavara
Copy link
Member

@pavelsavara and @fanyang-mono, should this issue be closed in favour of

You mean dotnet/runtime#89409

I think we should keep this one, because this one has some hits which are not just wasm debugger related.
Here we see other repo failures, on roslyn and aspnetcore CI.

if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

I have not investigated it myself yet. But got questions which could help to narrow it down

  • are there crash dump files on the machine ?
  • are there some hints on what's going on in the log files before crash ?
  • is this OOM or stack overflow in disguise ?
  • did we run out of disk space or number of inodes ?
  • hit some limit in docker container ?
  • is this always the same OS, version, image ?
  • could this be hardware issue ?

@fanyang-mono
Copy link
Member Author

This issue was originally created to track issues happened to wasm debugger tests (belonging to dotnet/runtime repo). However, the error message was generic enough to include issues from other repos, such as dotnet/roslyn and dotnet/aspnetcore. We need insights from owners from dotnet/roslyn and dotnet/aspnetcore to confirm that they are aware of these issue. And if they are okay to close this issue as well. We could close then.

@mitchdenny
Copy link
Member

We just got an instance of this in the dotnet/aspire repo. It wasn't a WASM related failure for us. Just looks like the test process crashed and took a few tests with it.

@nagilson
Copy link
Member

nagilson commented Dec 3, 2024

@marcpopMSFT was working on getting the test infrastructure to generate crash dumps when it times out to help with investigation. I think it was generating the dumps but not yet correctly uploading them to the artifacts.

We are waiting to hear from the infra team on an issue with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants