Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting
Environment
VSTS is being used for the deployment to a Windows web app on Azure using Service Principal
The Service Principal has contributor access on the resource group
- Agent — Hosted or Private:
Private Windows Agent is being used over an Internet proxy
Issue Description
The task is being used for deployment of a Windows Web App. The deployment goes through with in a min. But the task takes 8 to 9 mins to complete the execution. When debug logs are enabled we have observed that its failing on an attempt to update the deployment history and its re-attempting it 5 times after every 1 min. This is causing the delay of the execution and there is no way we can mask the task from attempting the deployment history update.
Error logs
2018-08-21T15:20:24.5754927Z ##[debug]build.sourceVersion=07c48833789f38ca37099893f39a1ec39bf7f4c9
2018-08-21T15:20:24.5755446Z ##[debug]build.repository.name=xxxxxxxxx
2018-08-21T15:20:24.5755939Z ##[debug]build.repository.provider=TfsGit
2018-08-21T15:20:24.5759274Z ##[debug][PUT]https://$WebAppName:@WebAppName.scm.domainname.com/api/deployments/781534864824574
2018-08-21T15:21:24.0974449Z ##[debug]Encountered a retriable error:ECONNRESET. Message: tunneling socket could not be established, statusCode=503.
2018-08-21T15:21:26.0995672Z ##[debug][PUT]https://$WebAppName:@WebAppName.scm.domainname.com/api/deployments/781534864824574
2018-08-21T15:22:26.0047963Z ##[debug]Encountered a retriable error:ECONNRESET. Message: tunneling socket could not be established, statusCode=503.
2018-08-21T15:22:32.0053097Z ##[debug][PUT]https://$WebAppName:@WebAppName.scm.domainname.com/api/deployments/781534864824574
2018-08-21T15:23:32.3024458Z ##[debug]Encountered a retriable error:ECONNRESET. Message: tunneling socket could not be established, statusCode=503.
2018-08-21T15:23:46.0096952Z ##[debug][PUT]https://$WebAppName:@WebAppName.scm.domainname.com/api/deployments/781534864824574
2018-08-21T15:24:46.0090902Z ##[debug]Encountered a retriable error:ECONNRESET. Message: tunneling socket could not be established, statusCode=503.
2018-08-21T15:25:16.0107429Z ##[debug][PUT]https://$WebAppName:***@WebAppName.scm.domainname.com/api/deployments/781534864824574
2018-08-21T15:26:16.0101812Z ##[debug]Processed: ##vso[task.logissue type=error;code=ECONNRESET;]
I have a for loop where I call 10-15 HTTP requests synchronously using npm request library. The URLs are the same except for one parameter. For example:
http://example.com/sendorder?orderid=1
http://example.com/sendorder?orderid=2
I call every request with different proxy addresses from my proxy pool. Out of 15 requests, 2 to 3 of them fails every time. I get two different errors. Sometimes the first one, sometimes the other. I also tried calling every request with the same proxy and I got the same error so the error is not related to some of the proxy addresses broken.
First type of error:
{ Error: tunneling socket could not be established, statusCode=503
at ClientRequest.onConnect (/Users/admin/Documents/node-binance-api/node_modules/tunnel-agent/index.js:166:19)
at Object.onceWrapper (events.js:319:30)
at emitThree (events.js:136:13)
at ClientRequest.emit (events.js:217:7)
at Socket.socketOnData (_http_client.js:474:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at TCP.onread (net.js:597:20) code: 'ECONNRESET' }
Second type of error:
{ Error: tunneling socket could not be established, cause=connect ETIMEDOUT 35.209.77.242:3128
at ClientRequest.onError (/Users/deniz/Documents/node-binance-api/node_modules/tunnel-agent/index.js:177:17)
at Object.onceWrapper (events.js:315:30)
at emitOne (events.js:116:13)
at ClientRequest.emit (events.js:211:7)
at Socket.socketErrorListener (_http_client.js:387:9)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at emitErrorNT (internal/streams/destroy.js:64:8)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9) code: 'ECONNRESET' }
This is production code and has been working without any problems for months. Now it started to get proxy connection errors. The proxies haven’t changed, the code hasn’t changed. I have been debugging for days but could not figure out what is causing these proxy connection errors. I tried with fresh proxy addresses, still the same.
Strange thing is, I tried waiting 10 seconds between each request, but still some of the requests fail with this error. So I don’t think it is because of the load on the proxy.
yarn install v1.21.1 warning package-lock.json found.
Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files.
To clear this warning, remove `package-lock.json`.
- [1/5] Validating package.json...
- [2/5] Resolving packages...
- [3/5] Fetching packages...
info fsevents@1.2.11: The platform "linux" is incompatible with this module.
info "fsevents@1.2.11" is an optional dependency and failed compatibility check. Excluding it from installation.
- [4/5] Linking dependencies...
warning " > @magento/eslint-config@1.5.2" has incorrect peer dependency "eslint-plugin-react-hooks@^1.6.0".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/core@~7.3.4".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/plugin-proposal-class-properties@~7.3.4".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/plugin-proposal-object-rest-spread@~7.3.4".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/plugin-syntax-dynamic-import@~7.2.0".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/plugin-syntax-jsx@~7.2.0".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/plugin-transform-react-jsx@~7.3.0".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/plugin-transform-runtime@~7.4.4".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/preset-env@~7.3.4".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "@babel/runtime@~7.4.2".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "babel-plugin-dynamic-import-node@~2.2.0".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "babel-plugin-graphql-tag@~2.0.0".
warning " > @magento/babel-preset-peregrine@1.0.1" has unmet peer dependency "react-hot-loader@~4.12.17".
warning " > graphql-cli-validate-magento-pwa-queries@1.2.0" has unmet peer dependency "graphql-cli@^3".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "@apollo/react-hooks@~3.1.2".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "@babel/runtime@~7.4.2".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "apollo-client@~2.6.4".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "informed@~2.11.17".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "react@~16.9.0".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "react-dom@~16.9.0".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "react-redux@~7.1.1".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "react-router-dom@~5.1.0".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "redux@~4.0.4".
warning " > @magento/peregrine@5.0.0" has unmet peer dependency "redux-actions@~2.6.5".
warning " > @magento/pwa-buildpack@5.0.0" has unmet peer dependency "babel-loader@~8.0.5".
warning " > @magento/pwa-buildpack@5.0.0" has unmet peer dependency "css-loader@~2.1.1".
warning " > @magento/pwa-buildpack@5.0.0" has unmet peer dependency "terser-webpack-plugin@~1.2.3".
warning " > @magento/pwa-buildpack@5.0.0" has unmet peer dependency "webpack@~4.38.0".
warning " > @magento/pwa-buildpack@5.0.0" has unmet peer dependency "workbox-webpack-plugin@~4.2.0".
warning " > @magento/upward-js@4.0.0" has unmet peer dependency "express@^4.16.4".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "apollo-cache-inmemory@~1.6.3".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "apollo-cache-persist@~0.1.1".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "apollo-client@2.6.4".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "apollo-link-http@~1.5.11".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "braintree-web-drop-in@~1.16.0".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "react@~16.9.0".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "react-redux@~7.1.1".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "react-router-dom@~5.1.0".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "redux@~4.0.1".
warning " > @magento/venia-ui@2.0.0" has unmet peer dependency "webpack@~4.41.2".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/pwa-buildpack > graphql-playground-middleware-express@1.7.12" has unmet peer dependency "express@^4.16.2".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-concept > react-hot-loader@4.12.18" has unmet peer dependency "@types/react@^15.0.0 || ^16.0.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-ui > @apollo/react-hooks@3.1.3" has unmet peer dependency "@types/react@^16.8.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-ui > @apollo/react-hooks@3.1.3" has unmet peer dependency "react-dom@^16.8.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-ui > react-head@3.1.1" has unmet peer dependency "react-dom@>=16.3".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-ui > react-slick@0.25.2" has unmet peer dependency "react-dom@^0.14.0 || ^15.0.1 || ^16.0.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-ui > @apollo/react-hooks > @apollo/react-common@3.1.3" has unmet peer dependency "@types/react@^16.8.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > @magento/venia-ui > @apollo/react-hooks > @apollo/react-common@3.1.3" has unmet peer dependency "apollo-utilities@^1.3.2".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > graphql-cli-validate-magento-pwa-queries > graphql-cli > apollo-codegen > apollo-codegen-core@0.20.1" has incorrect peer dependency "graphql@^0.11.0 || ^0.12.0 || ^0.13.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > graphql-cli-validate-magento-pwa-queries > graphql-cli > graphql-cli-prepare > graphql-import@0.4.5" has incorrect peer dependency "graphql@^0.11.0 || ^0.12.0 || ^0.13.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > graphql-cli-validate-magento-pwa-queries > graphql-cli > graphql-cli-prepare > graphql-static-binding@0.9.3" has incorrect peer dependency "graphql@^0.11.0 || ^0.12.0 || ^0.13.0".
warning "workspace-aggregator-4d7f1716-16a5-4d05-83b9-41e41f5ec283 > graphql-cli-validate-magento-pwa-queries > graphql-cli > tmp-graphql-config-extension-openapi > @kbrandwijk/swagger-to-graphql@2.4.3" has incorrect peer dependency "graphql@^0.12.0 || ^0.13.0".
- [5/5] Building fresh packages...
- [8/12] ⠂ iltorb
- [-/12] ⠂ waiting...
- [3/12] ⠂ iltorb
- [4/12] ⠄ sharp
error /home/aspire1030/pwa-studio/node_modules/sharp: Command failed.
Exit code: 1
Command: (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)
Arguments:
Directory: /home/aspire1030/pwa-studio/node_modules/sharp
Output:
info sharp Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.8.1/libvips-8.8.1-linux-x64.tar.gz
ERR! sharp tunneling socket could not be established, statusCode=503
info sharp Attempting to build from source via node-gyp but this may fail due to the above error
info sharp Please see https://sharp.pixelplumbing.com/page/install for required dependencies
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@10.16.3 | linux | x64
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/home/aspire1030/pwa-studio/packages/venia-concept/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/home/aspire1030/pwa-studio/node_modules/sharp/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/aspire1030/pwa-studio/packages/venia-concept/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/aspire1030/.node-gyp/10.16.3/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/aspire1030/.node-gyp/10.16.3',
gyp info spawn args '-Dnode_gyp_dir=/home/aspire1030/pwa-studio/packages/venia-concept/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/aspire1030/.node-gyp/10.16.3/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/home/aspire1030/pwa-studio/node_modules/sharp',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.' ]
make: Entering directory '/home/aspire1030/pwa-studio/node_modules/sharp/build'
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
TOUCH Release/obj.target/libvips-cpp.stamp
CXX(target) Release/obj.target/sharp/src/common.o
../src/common.cc:25:22: fatal error: vips/vips8: No such file or directory
compilation terminated.
sharp.target.mk:127: recipe for target 'Release/obj.target/sharp/src/common.o' failed
make: Leaving directory '/home/aspire1030/pwa-studio/node_modules/sharp/build'
make: *** [Release/obj.target/sharp/src/common.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/home/aspire1030/pwa-studio/packages/venia-concept/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack at ChildProcess.emit (events.js:198:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Linux 4.15.0-76-generic
gyp ERR! command "/usr/bin/node" "/home/aspire1030/pwa-studio/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /home/aspire1030/pwa-studio/node_modules/sharp