Pm2 перезапуск при ошибке - Исправление ошибок и поиск оптимальных решений проблем

Restart strategies

When starting application with PM2, application are automatically restarted on auto exit, event loop empty (node.js) or when application crash.
But you can also configure extra restart strategies like:

Restart app at a specified CRON time
Restart app when files have changed
Restart when app reach a memory threshold
Delay a start and automatic restart
Disable auto restart (app are always restarted with PM2) when crashing or exiting by default)
Restart application automatically at a specific exponential increasing time

Restart at cron time

Via CLI:

$ pm2 start app.js --cron-restart="0 0 * * *"
# Or when restarting an app
$ pm2 restart app --cron-restart="0 0 * * *"

Via configuration file, use the cron_restart attribute:

module.exports = {
  apps : [{
    name: 'Business News Watcher',
    script: 'app.js',
    instances: 1,
    cron_restart: '0 0 * * *',
    env: {
      NODE_ENV: 'development'
    },
    env_production: {
      NODE_ENV: 'production'
    }
  }]
}

To disable cron restart:

pm2 restart app --cron-restart 0

Restart on file change

PM2 can automatically restart your application when a file is modified in the current directory or its subdirectories:

Via CLI:

$ pm2 start app.js --watch

Note: If an application is started with the --watch option, stopping the app will not prevent it to be restarted on file change.
To totally disable the watch feature, do: pm2 stop app --watch or toggle the watch option on application restart via pm2 restart app --watch.

Via configuration file, use the watch: true attribute:

module.exports = {
  script: "app.js",
  watch: true
}

You can specify which folder to watch for change, ignore folder and watch files interval with these options:

module.exports = {
  script: "app.js",
  // Specify which folder to watch
  watch: ["server", "client"],
  // Specify delay between watch interval
  watch_delay: 1000,
  // Specify which folder to ignore 
  ignore_watch : ["node_modules", "client/img"],
}

Memory based restart strategy

PM2 allows to reload (auto fallback to restart if not in cluster) an application based on a memory limit/ Please note that the PM2 internal worker (which checks memory), starts every 30 seconds, so you may have to wait a bit before your process gets restarted automatically after reaching the memory threshold.

CLI:

$ pm2 start api.js --max-memory-restart 300M

Via configuration file, use the max_memory_restart attribute:

module.exports = {
  script: 'api.js',
  max_memory_restart: '300M'
}

Note: Units can be K(ilobyte) (e.g. 512K), M(egabyte) (e.g. 128M), G(igabyte) (e.g. 1G).

Restart Delay

Set a delay between auto restart with the Restart Delay strategy:

CLI:

$ pm2 start app.js --restart-delay=3000

Via configuration file, use the restart_delay attribute:

module.exports = {
  script: 'app.js',
  restart_delay: 3000
}

No Auto Restart

This is useful in case we wish to run 1-time scripts and don’t want the process manager to restart our script in case it’s completed running.

CLI:

$ pm2 start app.js --no-autorestart

Via configuration file, use the autorestart attribute:

module.exports = {
  script: 'app.js',
  autorestart: false
}

Skip Auto Restart For Specific Exit Codes

Sometimes you might want the application to automatically restart in case of failure (i.e. non-zero exit code),
while not wanting the process manager to restart it when it shuts down properly (i.e. exit code equal to 0).

In this case, you can still use PM2 just fine with a stop_exit_codes option set to exit codes that should skip auto restart:

CLI:

$ pm2 start app.js --stop-exit-codes 0

Or via configuration file, use the stop_exit_codes attribute:

module.exports = [{
  script: 'app.js',
  stop_exit_codes: [0]
}]

Exponential Backoff Restart Delay

A new restart mode has been implemented on PM2 Runtime, making your application restarts in a smarter way. Instead of restarting your application like crazy when exceptions happens (e.g. database is down), the exponential backoff restart will increase incrementally the time between restarts, reducing the pressure on your DB or your external provider… Pretty easy to use:

CLI:

$ pm2 start app.js --exp-backoff-restart-delay=100

Via configuration file, use the exp_backoff_restart_delay attribute:

module.exports = {
  script: 'app.js',
  exp_backoff_restart_delay: 100
}

When an application crash unexpectedly and the option --exp-backoff-restart-delay is activated, you will be able to see a new application status waiting restart.

By running pm2 logs you will also see the restart delay being incremented:

PM2      | App [throw:0] will restart in 100ms
PM2      | App [throw:0] exited with code [1] via signal [SIGINT]
PM2      | App [throw:0] will restart in 150ms
PM2      | App [throw:0] exited with code [1] via signal [SIGINT]
PM2      | App [throw:0] will restart in 225ms

As you can see the restart delay between restarts will increase in an exponential moving average, till reaching the maximum of 15000ms between restarts.

When the application will then get back to a stable mode (uptime without restarts of more than 30 seconds), the restart delay will automatically reset to 0ms.

Contribute to this page

Источник

Exponential Backoff Restart Delay

Available in PM2 >= 3.2

A new restart mode has been implemented on PM2 Runtime, making your application restarts in a smarter way. Instead of restarting your application like crazy when exceptions happens (e.g. database is down), the exponential backoff restart will increase incrementaly the time between restarts, reducing the pressure on your DB or your external provider… Pretty easy to use:

CLI:

$ pm2 start app.js --exp-backoff-restart-delay=100

Or via ecosystem.config.js file:

module.exports = [{
  script: 'app.js',
  exp_backoff_restart_delay: 100
}]

When an application crash unexpectedly and the option --exp-backoff-restart-delay is activated, you will be able to see a new application status waiting restart.

By running pm2 logs you will also see the restart delay being incremented:

PM2      | App [throw:0] will restart in 100ms
PM2      | App [throw:0] exited with code [1] via signal [SIGINT]
PM2      | App [throw:0] will restart in 150ms
PM2      | App [throw:0] exited with code [1] via signal [SIGINT]
PM2      | App [throw:0] will restart in 225ms

As you can see the restart delay between restarts will increase in an exponential moving average, till reaching the maximum of 15000ms between restarts.

When the application will then get back to a stable mode (uptime without restarts of more than 30 seconds), the restart delay will automatically reset to 0ms.

Fixed Restart Delay

Available in PM2 >= 0.9

You can also use the restart_delay to set a fixed timing between restarts:

CLI:

$ pm2 start app.js --restart-delay=3000

Or via ecosystem.config.js file:

module.exports = [{
  script: 'app.js',
  restart_delay: 3000
}]

Memory based reload strategy

Checkout https://pm2.io/docs/docs/runtime/features/memory-limit/

0second Downtime Reload

Checkout the cluster mode to get this behavior

No Auto Restart

This is useful in case we wish to run 1-time scripts and don’t want the process manager to restart our script in case it’s completed running.

Simply running these scripts from bash would terminate the script in case the ssh-session is terminated and the script should not get restarted when it completes execution.

PM2 is perfect for such cases, providing robust monitoring and logging

CLI:

$ pm2 start app.js --no-autorestart

Источник

In this article, we are going to learn, about restarting a Node.js application when an uncaught exception happens. For this, we are going to use the pm2 module.

Approach: Let’s see the approach step by step:

Step 1: Install the pm2 module and use it to start the server.
Step 2: When an uncaught exception happens, then execute the command process.exit() to stop the server.
Step 3: Then, pm2 module will automatically start the server again.

process.exit() stop the server and pm2 force it to start. In this way, the server will restart.

Implementation: Below is the step-by-step implementation of the above approach.

Step 1: Initializes NPM: Create and Locate your project folder in the terminal & type the command

npm init -y

It initializes our node application & makes a package.json file.

Step 2: Install Dependencies: Locate your root project directory into the terminal and type the command

npm install express pm2

To install express and pm2 as dependencies inside your project

Step 3: Creating a list of products: Let’s create an array of products and set it to constant products.

const products = [];

Step 4: Creating Routes for the home page and the products page: Let’s create two routes so that users can access the home page and the products page.

app.get('/', (req, res) => {
   res.send('Hello Geeks!');
});

app.get('/products', (req, res) => {
   if (products.length === 0) {
       res.send('No products found!');
       process.exit();
   } else {
       res.json(products);
   }
});

Inside the product route, we use process.exit() method to stop the server.

Complete Code:

Javascript

const express = require('express');

const app = express();

const products = [];

app.get('/', (req, res) => {

res.send('Hello Geeks!');

});

app.get('/products', (req, res) => {

if (products.length === 0) {

res.send('No products found!');

process.exit();

} else {

res.json(products);

}

});

app.listen(3000, ()=>{

console.log('listening on port 3000');

});

Steps to run the application: Inside the terminal type the command to run your script ‘app.js’ with pm2.

pm2 start app.js

Output:

Источник

PM2 production process manager

Quick start

Installation
Usage
Examples
Different ways to launch a process
Options
- Schema
How to update PM2?

Features

Transitional state of apps
Process listing
Automatic restart process based on memory
Monitoring CPU/Memory usage
Logs management
Clustering
Watch & Restart
Reloading without downtime
Make PM2 restart on server reboot
JSON app declaration
- Schema

Windows specifics

Windows

Deployment — ecosystem.json

Getting started with deployment
Deployment options
Considerations
Contributing

Using PM2 programmatically (via API)

Simple example
Programmatic API

Specific

Specific features
Configuration file
Enabling Harmony ES6
CoffeeScript
Testing PM2 on your prod environment
JSON app via pipe

Knowledge

Stateless apps ?
Transitional state of apps
Setup PM2 on server: tutorial
Logs and PID files
Execute any script: What is fork mode ?

Contributing/Development mode
Known bugs and workaround
They talk about it
License

Quick start

## Installation

The latest PM2 stable version is installable via NPM:

$ npm install pm2@latest -g

If the above fails use:

$ npm install git://github.com/Unitech/pm2#master -g

## Usage

Hello world:

## Raw Examples

# Fork mode
$ pm2 start app.js --name my-api # Name process

# Cluster mode
$ pm2 start app.js -i 0        # Will start maximum processes with LB depending on available CPUs
$ pm2 start app.js -i max      # Same as above, but deprecated yet.

# Listing

$ pm2 list               # Display all processes status
$ pm2 jlist              # Print process list in raw JSON
$ pm2 prettylist         # Print process list in beautified JSON

$ pm2 describe 0         # Display all informations about a specific process

$ pm2 monit              # Monitor all processes

# Logs

$ pm2 logs [--raw]       # Display all processes logs in streaming
$ pm2 ilogs              # Advanced termcaps interface to display logs
$ pm2 flush              # Empty all log file
$ pm2 reloadLogs         # Reload all logs

# Actions

$ pm2 stop all           # Stop all processes
$ pm2 restart all        # Restart all processes

$ pm2 reload all         # Will 0s downtime reload (for NETWORKED apps)
$ pm2 gracefulReload all # Send exit message then reload (for networked apps)

$ pm2 stop 0             # Stop specific process id
$ pm2 restart 0          # Restart specific process id

$ pm2 delete 0           # Will remove process from pm2 list
$ pm2 delete all         # Will remove all processes from pm2 list

# Misc

$ pm2 reset <process>    # Reset meta data (restarted time...)
$ pm2 updatePM2          # Update in memory pm2
$ pm2 ping               # Ensure pm2 daemon has been launched
$ pm2 sendSignal SIGUSR2 my-app # Send system signal to script
$ pm2 start app.js --no-daemon
$ pm2 start app.js --no-vizion
$ pm2 start app.js --no-autorestart

Different ways to launch a process

$ pm2 start app.js           # Start app.js

$ pm2 start app.js -- -a 23  # Pass arguments '-a 23' argument to app.js script

$ pm2 start app.js --name serverone # Start a process an name it as server one
                                    # you can now stop the process by doing
                                    # pm2 stop serverone

$ pm2 start app.js --node-args="--debug=7001" # --node-args to pass options to node V8

$ pm2 start app.js -i 0             # Start maximum processes depending on available CPUs (cluster mode)

$ pm2 start app.js --log-date-format "YYYY-MM-DD HH:mm Z"    # Log will be prefixed with custom time format

$ pm2 start app.json                # Start processes with options declared in app.json
                                    # Go to chapter Multi process JSON declaration for more

$ pm2 start app.js -e err.log -o out.log  # Start and specify error and out log

For scripts in other languages:

$ pm2 start echo.pl --interpreter=perl

$ pm2 start echo.coffee
$ pm2 start echo.php
$ pm2 start echo.py
$ pm2 start echo.sh
$ pm2 start echo.rb

The interpreter is set by default with this equivalence:

{
  ".sh": "bash",
  ".py": "python",
  ".rb": "ruby",
  ".coffee" : "coffee",
  ".php": "php",
  ".pl" : "perl",
  ".js" : "node"
}

## Options

Options:

   -h, --help                           output usage information
   -V, --version                        output the version number
   -v --version                         get version
   -s --silent                          hide all messages
   -m --mini-list                       display a compacted list without formatting
   -f --force                           force actions
   -n --name <name>                     set a <name> for script
   -i --instances <number>              launch [number] instances (for networked app)(load balanced)
   -l --log [path]                      specify entire log file (error and out are both included)
   -o --output <path>                   specify out log file
   -e --error <path>                    specify error log file
   -p --pid <pid>                       specify pid file
   --max-memory-restart <memory>        specify max memory amount used to autorestart (in megaoctets)
   --env <environment_name>             specify environment to get specific env variables (for JSON declaration)
   -x --execute-command                 execute a program using fork system
   -u --user <username>                 define user when generating startup script
   -c --cron <cron_pattern>             restart a running process based on a cron pattern
   -w --write                           write configuration in local folder
   --interpreter <interpreter>          the interpreter pm2 should use for executing app (bash, python...)
   --log-date-format <momentjs format>  add custom prefix timestamp to logs
   --no-daemon                          run pm2 daemon in the foreground if it doesn't exist already
   --merge-logs                         merge logs from different instances but keep error and out separated
   --watch                              watch application folder for changes
   --ignore-watch <folders|files>       folder/files to be ignored watching, chould be a specific name or regex - e.g. --ignore-watch="test node_modules "some scripts""
   --node-args <node_args>              space delimited arguments to pass to node in cluster mode - e.g. --node-args="--debug=7001 --trace-deprecation"
   --no-color                           skip colors
   --no-vizion                          skip vizion features (versioning control)
   --no-autorestart                     do not automatically restart apps

## How to update PM2

Install the latest pm2 version :

$ npm install pm2@latest -g

Then update the in-memory PM2 :

Features

## Managing applications states

PM2 is a process manager.
It manages your applications states, so you can start, stop, restart and delete processes.

Start a process:

$ pm2 start app.js --name "my-api"
$ pm2 start web.js --name "web-interface"

Now let’s say I need to stop the web-interface:

As you can see the process hasn’t disappeared. It’s still there but in stopped status.

To restart it just do:

$ pm2 restart web-interface

Now I want to delete the app from the PM2 process list.
To do so:

$ pm2 delete web-interface

## Process listing

To list all running processes:

$ pm2 list
# Or
$ pm2 [list|ls|l|status]

To get more details about a specific process:

## Monitoring CPU/Memory

PM2 gives you a simple way to monitor the resource usage of your application.
You can monitor memory and cpu very easily, straight from your terminal:

Please note that Keymetrics allows to monitor applications much better, from errors, performances, vital signs and much more.

## Clustering (cluster_mode)

The cluster mode allows to scale your Node.js application accross all CPUs available and to update them without any downtime!

It’s perfectly fitted for networked applications handling HTTP(s)/UDP/TCP connections.

To enable the cluster mode, just pass the -i option:

You can pass multiple values to the instances (-i) option:

# Start the maximum processes depending on available CPUs
$ pm2 start app.js -i 0

# Start the maximum processes -1 depending on available CPUs
$ pm2 start app.js -i -1

# Start 3 processes
$ pm2 start app.js -i 3

Considerations

You don’t need to modify anything from you code to be able to use this nifty feature.
In your application the environment variable NODE_APP_INSTANCE is exposed so you can listen for different port if needed. (e.g .listen(8000 + process.env.NODE_APP_INSTANCE))
Be sure your application is stateless meaning that there is not any local data stored in the process, like sessions/websocket connections etc. Use Redis, Mongo or other DB to share states between processes

Hot reload / 0s reload

As opposed to restart, which kills and restarts the process, reload achieves a 0-second-downtime reload.

Warning This feature only works for apps in cluster mode, that uses HTTP/HTTPS/Socket connections.

To reload an app:

If the reload system hasn’t managed to reload your app, a timeout will fallback to a classic restart.

Graceful reload

Sometimes you can experience a very long reload, or a reload that doesn’t work (fallback to restart) meaning that your app still has open connections on exit.
Or you may need to close all databases connections, clear a data queue or whatever.

To work around this problem you have to use the graceful reload.

Graceful reload is a mechanism that will send a shutdown message to your process before reloading it.
You can control the time that the app has to shutdown via the PM2_GRACEFUL_TIMEOUT environment variable.

Example:

process.on('message', function(msg) {
  if (msg == 'shutdown') {
    // Your process is going to be reloaded
    // You have to close all database/socket.io/* connections

    console.log('Closing all connections...');

    // You will have 4000ms to close all connections before
    // the reload mechanism will try to do its job

    setTimeout(function() {
      console.log('Finished closing connections');
      // This timeout means that all connections have been closed
      // Now we can exit to let the reload mechanism do its job
      process.exit(0);
    }, 1500);
  }
});

Then use the command:

$ pm2 gracefulReload [all|name]

When PM2 starts a new process to replace an old one, it will wait for the new process to begin listening to a connection or a timeout before sending the shutdown message to the old one. You can define the timeout value with the PM2_GRACEFUL_LISTEN_TIMEOUT environament variable. If a script does not need to listen to a connection, it can manually tell PM2 that the process has started up by calling process.send('online').

## Logs management

PM2 allows you to manage logs easily. You can display all applications logs in real-time, flush and reload them.
There are also differents way to configure how PM2 should handle your logs (separated in differents files, merged, with timestamp…) without modifying anything from your code.

Displaying logs in real-time

Displaying logs of specified process or all processes in real-time:

$ pm2 logs
$ pm2 logs big-api

Flushing logs

$ pm2 flush # Clear all the logs

You can configure a Cron Job to call this command every X days.
Or you can install Log rotate to handle the log rotation.

Reloading all logs

Reloading logs is specially usefull for Logrotate or any other rotating log system.

You can reload logs by sending SIGUSR2 to the PM2 process.

You can also reload all logs via the command line with:

Configuring logs

CLI

$ pm2 start echo.js --merge-logs --log-date-format="YYYY-MM-DD HH:mm Z"

Options:

--merge-logs                 do not postfix log file with process id
--log-date-format <format>   prefix logs with formated timestamp
-l --log [path]              specify entire log file (error and out are both included)
-o --output <path>           specify out log file
-e --error <path>            specify error log file

JSON / Programmatic

{
  "script"        : "echo.js",
  "err_file"      : "err.log",
  "out_file"      : "out.log",
  "merge_logs"    : true,
  log_date_format : "YYYY-MM-DD HH:mm Z"
}

Note: To merge all logs into the same file set the same value for err_file, out_file.

## Max Memory Restart

PM2 allows to restart an application based on a memory limit.

CLI

$ pm2 start big-array.js --max-memory-restart 20M

JSON

{
  "name"   : "max_mem",
  "script" : "big-array.js",
  "max_memory_restart" : "20M"
}

Programmatic

pm2.start({{
  name               : "max_mem",
  script             : "big-array.js",
  max_memory_restart : "20M"
}, function(err, proc) {
  // Processing
});

Units

Units can be K(ilobyte), M(egabyte), G(igabyte).

## Startup script

PM2 can generate startup scripts and configure them.
PM2 is also smart enough to save all your process list and to bring back all your processes at machine restart.

$ pm2 startup
# auto-detect platform
$ pm2 startup [platform]
# render startup-script for a specific platform, the [platform] could be one of:
#   ubuntu|centos|redhat|gentoo|systemd|darwin|amazon

Once you have started the apps and want to keep them on server reboot do:

Warning It’s tricky to make this feature work generically, so once PM2 has setup your startup script, reboot your server to make sure that PM2 has launched your apps!

Startup Systems support

Three types of startup scripts are available:

SystemV init script (with the option ubuntu or centos)
OpenRC init script (with the option gentoo)
SystemD init script (with the systemd option)

The startup options are using:

ubuntu will use updaterc.d and the script lib/scripts/pm2-init.sh
centos/redhat will use chkconfig and the script lib/scripts/pm2-init-centos.sh
gentoo will use rc-update and the script lib/scripts/pm2
systemd will use systemctl and the script lib/scripts/pm2.service
darwin will use launchd to load a specific plist to resurrect processes after reboot.

User permissions

Let’s say you want the startup script to be executed under another user.

Just use the -u <username> option !

$ pm2 startup ubuntu -u www

Related commands

Dump all processes status and environment managed by PM2:

It populates the file ~/.pm2/dump.pm2 by default.

To bring back the latest dump:

Watch & Restart

PM2 can automatically restart your app when a file changes in the current directory or its subdirectories:

$ pm2 start app.js --watch

If --watch is enabled, stopping it won’t stop watching:

pm2 stop 0 will not stop watching
pm2 stop --watch 0 will stop watching

Restart toggle the watch parameter when triggered.

To watch specific paths, please use a JSON app declaration, watch can take a string or an array of paths. Default is true:

{
  "watch": ["server", "client"],
  "ignore_watch" : ["node_modules", "client/img"],
  "watch_options": {
    "followSymlinks": false
  }
}

As specified in the Schema:

watch can be a boolean, an array of paths or a string representing a path. Default to false
ignore_watch can be an array of paths or a string, it’ll be interpreted by chokidar as a glob or a regular expression.
watch_options is an object that will replace options given to chokidar. Please refer to chokidar documentation for the definition.

PM2 is giving chokidar these Default options:

var watch_options = {
  persistent    : true,
  ignoreInitial : true
}

JSON app declaration

PM2 empowers your process management workflow, by allowing you to fine-tune the behavior, options, environment variables, logs files… of each process you need to manage via JSON configuration.

It’s particularly usefull for micro service based applications.

You can define parameters for your apps in a JSON file:

{
  "apps" : [{
    "name"        : "worker-app",
    "script"      : "worker.js",
    "args"        : ["--toto=heya coco", "-d", "1"],
    "watch"       : true,
    "node_args"   : "--harmony",
    "merge_logs"  : true,
    "cwd"         : "/this/is/a/path/to/start/script",
    "env": {
        "NODE_ENV": "production",
        "AWESOME_SERVICE_API_TOKEN": "xxx"
    }
  },{
    "name"       : "api-app",
    "script"     : "api.js",
    "instances"  : 4,
    "exec_mode"  : "cluster_mode",
    "error_file" : "./examples/child-err.log",
    "out_file"   : "./examples/child-out.log",
    "pid_file"   : "./examples/child.pid"
  }]
}

Then you can run:

# Start all apps
$ pm2 start processes.json

# Stop
$ pm2 stop processes.json

# Delete from PM2
$ pm2 delete processes.json

# Restart all
$ pm2 restart processes.json

Options

The following are valid options for JSON app declarations:

{
  "name"             : "node-app",
  "cwd"              : "/srv/node-app/current",
  "args"             : ["--toto=heya coco", "-d", "1"],
  "script"           : "bin/app.js",
  "node_args"        : ["--harmony", " --max-stack-size=102400000"],
  "log_date_format"  : "YYYY-MM-DD HH:mm Z",
  "error_file"       : "/var/log/node-app/node-app.stderr.log",
  "out_file"         : "log/node-app.stdout.log",
  "pid_file"         : "pids/node-geo-api.pid",
  "instances"        : 6, //or 0 => 'max'
  "min_uptime"       : "200s", // 200 seconds, defaults to 1000
  "max_restarts"     : 10, // defaults to 15
  "max_memory_restart": "1M", // 1 megabytes, e.g.: "2G", "10M", "100K", 1024 the default unit is byte.
  "cron_restart"     : "1 0 * * *",
  "watch"            : false,
  "ignore_watch"      : ["[\/\\]\./", "node_modules"],
  "merge_logs"       : true,
  "exec_interpreter" : "node",
  "exec_mode"        : "fork",
  "autorestart"      : false, // enable/disable automatic restart when an app crashes or exits
  "vizion"           : false, // enable/disable vizion features (versioning control)
  "env": {
    "NODE_ENV": "production",
    "AWESOME_SERVICE_API_TOKEN": "xxx"
  }
}

Schema

The completely definitions:

{
  "script": {
    "type": "string",
    "require": true
  },
  "args": {
    "type": [
      "array",
      "string"
    ]
  },
  "node_args": {
    "type": [
      "array",
      "string"
    ]
  },
  "name": {
    "type": "string"
  },
  "max_memory_restart": {
    "type": [
      "string",
      "number"
    ],
    "regex": "^\d+(G|M|K)?$",
    "ext_type": "sbyte",
    "desc": "it should be a NUMBER - byte, "[NUMBER]G"(Gigabyte), "[NUMBER]M"(Megabyte) or "[NUMBER]K"(Kilobyte)"
  },
  "instances": {
    "type": "number",
    "min": 0
  },
  "log_file": {
    "type": [
      "boolean",
      "string"
    ],
    "alias": "log"
  },
  "error_file": {
    "type": "string",
    "alias": "error"
  },
  "out_file": {
    "type": "string",
    "alias": "output"
  },
  "pid_file": {
    "type": "string",
    "alias": "pid"
  },
  "cron_restart": {
    "type": "string",
    "alias": "cron"
  },
  "cwd": {
    "type": "string"
  },
  "merge_logs": {
    "type": "boolean"
  },
  "watch": {
    "type": "boolean"
  },
  "ignore_watch": {
    "type": [
      "array",
      "string"
    ]
  },
  "watch_options": {
    "type": "object"
  },
  "env": {
    "type": ["object", "string"]
  },
  "^env_\S*$": {
    "type": [
      "object",
      "string"
    ]
  },
  "log_date_format": {
    "type": "string"
  },
  "min_uptime": {
    "type": [
      "number",
      "string"
    ],
    "regex": "^\d+(h|m|s)?$",
    "desc": "it should be a NUMBER - milliseconds, "[NUMBER]h"(hours), "[NUMBER]m"(minutes) or "[NUMBER]s"(seconds)",
    "min": 100,
    "ext_type": "stime"
  },
  "max_restarts": {
    "type": "number",
    "min": 0
  },
  "exec_mode": {
    "type": "string",
    "regex": "^(cluster|fork)(_mode)?$",
    "alias": "executeCommand",
    "desc": "it should be "cluster"("cluster_mode") or "fork"("fork_mode") only"
  },
  "exec_interpreter": {
    "type": "string",
    "alias": "interpreter"
  },
  "write": {
    "type": "boolean"
  },
  "force": {
    "type": "boolean"
  }
}

Considerations

All command line options passed when using the JSON app declaration will be dropped i.e.

$ cat node-app-1.json

{
  "name" : "node-app-1",
  "script" : "app.js",
  "cwd" : "/srv/node-app-1/current"
}

JSON app declarations are additive. Continuing from above:

$ pm2 start node-app-2.json
$ ps aux | grep node-app
root  14735  5.8  1.1  752476  83932 ? Sl 00:08 0:00 pm2: node-app-1
root  24271  0.0  0.3  696428  24208 ? Sl 17:36 0:00 pm2: node-app-2

Note that if you execute pm2 start node-app-2 again, it will spawn an additional instance node-app-2.

cwd: your JSON declaration does not need to reside with your script. If you wish to maintain the JSON(s) in a location other than your script (say, /etc/pm2/conf.d/node-app.json) you will need to use the cwd feature. (Note, this is especially helpful for capistrano style directory structures that use symlinks.) Files can be either relative to the cwd directory, or absolute (example below.)
All the keys can be used in a JSON configured file, and just need to make a small change in CLI, e.g.:

exec_interpreter  -> --interpreter
exec_mode         -> --execute_command
max_restarts      -> --max_restarts
force             -> --force

If the alias exists, you can using it as a CLI option, but do not forget to turn the camelCasing to underscore split — executeCommand to --execute_command.

Notes

Using quote to make an ESC, e.g.:

$pm2 start test.js --node-args "port=3001 sitename='first pm2 app'"

The nodeArgs will be

[
  "port=3001",
  "sitename=first pm2 app"
]

But not

[
  "port=3001",
  "sitename='first",
  "pm2",
  "app'"
]

RegExp key

Matches the keys of configured JSON by RegExp but not a specific String, e.g. ^env_\S*$ will match all env keys like env_production, env_test, and make sure the values conform to the schemas.
Special ext_type
- min_uptime
  
  Value of min_uptime could be:
  - Number
    e.g. "min_uptime": 3000 means 3000 milliseconds.
  - String
    In the meantime we are making it briefness and easy configuration: h, m and s, e.g.: "min_uptime": "1h" means one hour, "min_uptime": "5m" means five minutes and "min_uptime": "10s" means ten seconds (At last, it will be transformed into milliseconds).
- max_memory_restart
  
  Value of max_memory_restart could be:
  - Number
    e.g. "max_memory_restart": 1024 means 1024 bytes (NOT BITS).
  - String
    In the meantime we are making it briefness and easy configuration: G, M and K, e.g.: "max_memory_restart": "1G" means one gigabytes, "max_memory_restart": "5M" means five megabytes and "max_memory_restart": "10K" means ten kilobytes (At last, it will be transformed into byte(s)).
Optional values

Like exec_mode, value could be one of cluster (cluster_mode) or fork (fork_mode) only.
Should known
- maximum
  
  "instances": 0 means starting maximum processes depending on available CPUs (cluster mode)
- array
  
  args, node_args and ignore_watch could be type of Array (e.g.: "args": ["--toto=heya coco", "-d", "1"]) or string (e.g.: "args": "--to='heya coco' -d 1")

# Windows

Make sure you have tail.exe in your path, confirm using «where»

C:>where tail
C:Program Files (x86)Gitbintail.exe

Tail can be found as part of Git, Cygwin and MingW packages. Tail needs to be able to support «-f» and «-n» options.

# Deployment

PM2 embed a simple and powerful deployment system with revision tracing.
It’s based on https://github.com/visionmedia/deploy

A step-by-step tutorial is available here : Deploy and Iterate faster with PM2 deploy

Getting started with deployment

Please read the Considerations to use PM2 deploy

1- Generate a sample ecosystem.json file that list processes and deployment environment

In the current folder a ecosystem.json file will be created.
It contains this:

{
  "apps" : [{
    "name"      : "API",
    "script"    : "app.js",
    "env": {
      "COMMON_VARIABLE": "true"
    },
    "env_production" : {
      "NODE_ENV": "production"
    }
  },{
    "name"      : "WEB",
    "script"    : "web.js"
  }],
  "deploy" : {
    "production" : {
      "user" : "node",
      "host" : "212.83.163.1",
      "ref"  : "origin/master",
      "repo" : "git@github.com:repo.git",
      "path" : "/var/www/production",
      "post-deploy" : "pm2 startOrRestart ecosystem.json --env production"
    },
    "dev" : {
      "user" : "node",
      "host" : "212.83.163.1",
      "ref"  : "origin/master",
      "repo" : "git@github.com:repo.git",
      "path" : "/var/www/development",
      "post-deploy" : "pm2 startOrRestart ecosystem.json --env dev",
      "env"  : {
        "NODE_ENV": "dev"
      }
    }
  }
}

Edit the file according to your needs.

2- Be sure that you have the public ssh key on your local machine

$ ssh-keygen -t rsa
$ ssh-copy-id root@myserver.com

3- Now initialize the remote folder with:

$ pm2 deploy <configuration_file> <environment> setup

E.g:

$ pm2 deploy ecosystem.json production setup

This command will create all the folders on your remote server.

4- Deploy your code

$ pm2 deploy ecosystem.json production

Now your code will be populated, installed and started with PM2

## Deployment options

Display deploy help via pm2 deploy help:

$ pm2 deploy <configuration_file> <environment> <command>

  Commands:
    setup                run remote setup commands
    update               update deploy to the latest release
    revert [n]           revert to [n]th last deployment or 1
    curr[ent]            output current release commit
    prev[ious]           output previous release commit
    exec|run <cmd>       execute the given <cmd>
    list                 list previous deploy commits
    [ref]                deploy to [ref], the "ref" setting, or latest tag

Commands

$ pm2 startOrRestart all.json            # Invoke restart on all apps in JSON
$ pm2 startOrReload all.json             # Invoke reload
$ pm2 startOrGracefulReload all.json     # Invoke gracefulReload

Using file key for authenticating

Just add the «key» attribute with file path to the .pem key within the attributes «user», «hosts»…

    "production" : {
      "key"  : "/path/to/some.pem",
      "user" : "node",
      "host" : "212.83.163.1",
      "ref"  : "origin/master",
      "repo" : "git@github.com:repo.git",
      "path" : "/var/www/production",
      "post-deploy" : "pm2 startOrRestart ecosystem.json --env production"
    },

## Considerations

You might want to commit your node_modules folder (#622) or add the npm install command to the post-deploy section: "post-deploy" : "npm install && pm2 startOrRestart ecosystem.json --env production"
Verify that your remote server has the permission to git clone the repository
You can declare specific environment variable depending on the environment you want to deploy the code to. For instance to declare variables for the production environment, just add «env_production»: {} and declare that variables.
PM2 will look by default to ecosystem.json. So you can skip the <configuration_file> options if it’s the case
You can embed the «apps» & «deploy» section in the package.json
It deploys your code via ssh, you don’t need any dependencies
Process are initialized / started automatically depending on application name in ecosystem.json
PM2-deploy repository is there: pm2-deploy

## Contributing

The module is https://github.com/Unitech/pm2-deploy
Feel free to PR for any changes or fix.

# Using PM2 programmatically

PM2 can be used programmatically, meaning that you can embed a process manager directly in your code, spawn processes, keep them alive even if the main script is exited.

Check out this article for more informations.

Simple example

This will require pm2, launch test.js, list processes then exit the script.
You will notice that after exiting this script you will be able to see test.js process with pm2 list

var pm2 = require('pm2');

// Connect or launch PM2
pm2.connect(function(err) {

  // Start a script on the current folder
  pm2.start('test.js', { name: 'test' }, function(err, proc) {
    if (err) throw new Error('err');

    // Get all processes running
    pm2.list(function(err, process_list) {
      console.log(process_list);

      // Disconnect to PM2
      pm2.disconnect(function() { process.exit(0) });
    });
  });
})

## Programmatic API

Method name	API
Connect/Launch	pm2.connect(fn(err){})
Disconnect	pm2.disconnect(fn(err, proc){})

Consideration with .connect: the .connect method connect to the local PM2, but if PM2 is not up, it will launch it and will put in in background as you launched it via CLI.

<tr>
  <td><b>Reload</b></td>
  <td>pm2.reload(proc_name|all, fn(err, proc){})</td>
</tr>
  <tr>
  <td><b>Graceful Reload</b></td>
  <td>pm2.gracefulReload(proc_name|all, fn(err, proc){})</td>
</tr>

Method name	API
Start	pm2.start(script_path\|json_object\|json_path, options, fn(err, proc){})
Options	nodeArgs(arr), scriptArgs(arr), name(str), instances(int), error(str), output(str), pid(str), cron(str), mergeLogs(bool), watch(bool), runAsUser(int), runAsGroup(int), executeCommand(bool), interpreter(str), write(bool)
Restart	pm2.restart(proc_name\|proc_id\|all, fn(err, proc){})
Stop	pm2.stop(proc_name\|proc_id\|all, fn(err, proc){})
Delete	pm2.delete(proc_name\|proc_id\|all, fn(err, proc){})

Method name	API
List	pm2.list(fn(err, list){})
Describe process	pm2.describe(proc_name\|proc_id, fn(err, list){})
Dump (save)	pm2.dump(fn(err, ret){})
Flush logs	pm2.flush(fn(err, ret){})
Reload logs	pm2.reloadLogs(fn(err, ret){})
Send signal	pm2.sendSignalToProcessName(signal,proc,fn(err, ret){})
Generate start script	pm2.startup(platform, fn(err, ret){})
Kill PM2	pm2.killDaemon(fn(err, ret){})

# Special features

Launching PM2 without daemonizing itself:

$ pm2 start app.js --no-daemon

Sending a system signal to a process:

$ pm2 sendSignal SIGUSR2 my-app

## Configuration file

You can specify the following options by editing the file ~/.pm2/custom_options.sh:

PM2_RPC_PORT
PM2_PUB_PORT
PM2_BIND_ADDR
PM2_API_PORT
PM2_GRACEFUL_TIMEOUT
PM2_MODIFY_REQUIRE
PM2_KILL_TIMEOUT

API health endpoint

## Enabling Harmony ES6

The --node-args option permit to launch script with V8 flags, so to enable harmony for a process just do this:

$ pm2 start my_app.js --node-args="--harmony"

And with JSON declaration:

[{
  "name" : "ES6",
  "script" : "es6.js",
  "node_args" : "--harmony"
}]

## CoffeeScript

$ pm2 start server.coffee --interpreter coffee

That’s all!

Knowledge

## Stateless apps

We recommend (and you must) write stateless NodeJS apps. Apps that don’t retain any form of local variables or local instances or whatever local.
Every data, states, websocket session, session data, must be shared via any kind of database.

We recommend using Redis for sharing session data, websocket.

SocketIO with Redis : [https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO](Configuring SocketIO)
Redis session store for Connect : https://github.com/visionmedia/connect-redis

We recommend following the 12 factor convention : http://12factor.net/

## Setup pm2 on a server

How To Use pm2 to Setup a Node.js Production Environment On An Ubuntu VPS

## Log and PID files

By default, logs (error and output), pid files, dumps, and PM2 logs are located in ~/.pm2/:

.pm2/
├── dump.pm2
├── custom_options.sh
├── pm2.log
├── pm2.pid
├── logs
└── pids

## Execute any kind of script

In fork mode almost all options are the same as the cluster mode. But there is no reload or gracefulReload command.

You can also exec scripts written in other languages:

$ pm2 start my-bash-script.sh -x --interpreter bash

$ pm2 start my-python-script.py -x --interpreter python

The interpreter is deduced from the file extension from the following list.

JSON configuration

To run a non-JS interpreter you must set exec_mode to fork_mode and exec_interpreter to your interpreter of choice.
For example:

{
  "apps" : [{
    "name"       : "bash-worker",
    "script"     : "./a-bash-script",
    "exec_interpreter": "bash",
    "exec_mode"  : "fork_mode"
  }, {
    "name"       : "ruby-worker",
    "script"     : "./some-ruby-script",
    "exec_interpreter": "ruby",
    "exec_mode"  : "fork_mode"
  }]
}

## JSON app configuration via pipe from stdout

Pull-requests:

#273
#279

#!/bin/bash

read -d '' my_json <<_EOF_
[{
    "name"       : "app1",
    "script"     : "/home/projects/pm2_nodetest/app.js",
    "instances"  : "4",
    "error_file" : "./logz/child-err.log",
    "out_file"   : "./logz/child-out.log",
    "pid_file"   : "./logz/child.pid",
    "exec_mode"  : "cluster_mode",
    "port"       : 4200
}]
_EOF_

echo $my_json | pm2 start -

## Is my production server ready for PM2?

Just try the tests before using PM2 on your production server

$ git clone https://github.com/Unitech/pm2.git
$ cd pm2
$ npm install  # Or do NODE_ENV=development npm install if some packages are missing
$ npm test

If a test is broken please report us issues here
Also make sure you have all dependencies needed. For Ubuntu:

$ sudo apt-get install build-essential
# nvm is a Node.js version manager - https://github.com/creationix/nvm
$ wget -qO- https://raw.github.com/creationix/nvm/master/install.sh | sh
$ nvm install v0.11.14
$ nvm use v0.11.14
$ nvm alias default v0.11.14

## Contributing/Development mode

To hack PM2, it’s very simple:

$ pm2 kill   # kill the current pm2
$ git clone my_pm2_fork.git
$ cd pm2/
$ DEBUG=* PM2_DEBUG=true ./bin/pm2 --no-daemon

Each time you edit the code, be sure to kill and restart PM2 to make changes taking effect.

Install PM2 development

$ npm install git://github.com/Unitech/pm2#development -g

## Known bugs and workarounds

First, install the lastest PM2 version:

$ npm install -g pm2@latest

Node 0.10.x doesn’t free the script port when stopped in cluster_mode

Don’t use the cluster_mode via -i option.

User tips from issues

Vagrant and pm2 #289
Start the same app on different ports #322
Using ansible with pm2
Cron string as argument
Restart when process reaches a specific memory amount
Sticky sessions and socket.io discussion
EACCESS — understanding pm2 user/root rights

## External resources and articles

Goodbye node-forever, hello pm2
https://serversforhackers.com/editions/2014/11/04/pm2/
http://www.allaboutghost.com/keep-ghost-running-with-pm2/
http://blog.ponyfoo.com/2013/09/19/deploying-node-apps-to-aws-using-grunt
http://www.allaboutghost.com/keep-ghost-running-with-pm2/
http://bioselemental.com/keeping-ghost-alive-with-pm2/
http://blog.chyld.net/installing-ghost-on-ubuntu-13-10-aws-ec2-instance-with-pm2/
http://blog.marvinroger.fr/gerer-ses-applications-node-en-production-pm2/
https://www.codersgrid.com/2013/06/29/pm2-process-manager-for-node-js/
http://www.z-car.com/blog/programming/how-to-rotate-logs-using-pm2-process-manager-for-node-js
http://yosoftware.com/blog/7-tips-for-a-node-js/
https://www.exponential.io/blog/nodeday-2014-moving-a-large-developer-workforce-to-nodejs
http://blog.rapsli.ch/posts/2013/2013-10-17-node-monitor-pm2.html
https://coderwall.com/p/igdqyw
http://revdancatt.com/2013/09/17/node-day-1-getting-the-server-installing-node-and-pm2/
https://medium.com/tech-talk/e7c0b0e5ce3c

Contributors

Restart the application using a scheduled task

Use the following command to set the task of scheduled restart

$ pm2 start server.js --cron-restart="0 0 * * *"
# Or set a scheduled task when restarting
$ pm2 restart app --cron-restart="0 0 * * *"

If yes configuration file If so, use cron_restart attribute:

server.config.js

module.exports = {
  apps : [{
    name: 'server',
    script: 'server.js',
    instances: 1,
    cron_restart: '0 0 * * *',
    env: {
      NODE_ENV: 'development'
    },
    env_production: {
      NODE_ENV: 'production'
    }
  }]
}

Automatically restart the application after file changes

When the files in the current directory or its subdirectory are modified, PM2 can automatically restart your application:

Use the following command to start the application by specifying the option — watch later

$ pm2 start server.js --watch

Let’s illustrate this situation through gif dynamic diagram

Note: if the application starts using the – watch option, stopping the application does not prevent it from restarting when the file changes. To completely disable the watch function, execute the following command:

$ pm2 stop app --watch

Or use the following command to switch the watch option when the application restarts.

$ pm2 restart app --watch

In the configuration file, use the watch: true attribute

module.exports = {
  script: "server.js",
  watch: true
}

We can also specify which folder to listen to in the configuration file, and automatically restart the application when its contents are modified. You can also specify that some folders are ignored, and no restart will be triggered regardless of how their contents change

module.exports = {
  script: "server.js",
  // Specify the folder to listen to
  watch: ["server", "client"],
  // Specify delay time
  watch_delay: 1000,
  // Specify the folder to ignore
  ignore_watch : ["node_modules", "client/img"],
}

Restart when application reaches memory threshold

PM2 allows applications to be reloaded according to the memory limit (if they are not in the cluster, they will automatically fall back and restart) / please note that PM2 internal working program (check memory) starts every 30 seconds, so it may take a moment after reaching the memory threshold, and the process will restart automatically.

Use the — Max memory restart option to specify the memory threshold.

$ pm2 start server.js --max-memory-restart 300M

Use Max in configuration file_ memory_ Restart attribute.

server.config.js

module.exports = {
  script: 'server.js',
  max_memory_restart: '300M'
}

Note: the units can be K(ilobyte) (e.g. 512K), m (egabyte) (e.g. 128M), G (igabyte) (e.g. 1G).

Delayed restart

Use the Restart Delay policy to set the delay between automatic restarts:

$ pm2 start server.js --restart-delay=3000

In the configuration file, use restart_ The delay property sets delayed restart.

server.config.js

module.exports = {
  script: 'server.js',
  restart_delay: 3000
}

Prohibit automatic restart

This is useful if we want to run the script once and do not want the process manager to restart our script when the script finishes running.

$ pm2 start server.js --no-autorestart

In the configuration file, use the autorestart: false attribute to disable automatic restart.

server.config.js

module.exports = {
  script: 'server.js',
  autorestart: false
}

Specifies an exit code that does not restart automatically

Sometimes we may want the application to restart automatically in the event of a failure (that is, a non-zero exit code) rather than the process manager to restart it when it closes properly (that is, the exit code is equal to 0).

In this case, PM2 can still be used well and stop_ exit_ The codes option is set to the exit code that should skip automatic restart:

$ pm2 start server.js --stop-exit-codes 0

In the configuration file, use stop_ exit_ The codes property sets the exit code without automatic restart.

server.config.js

module.exports = [{
  script: 'server.js',
  stop_exit_codes: [0]
}]

Exponential backoff restart delay

A new restart mode has been implemented on PM2 Runtime to restart our applications in a more intelligent way. When an exception occurs (such as database shutdown), instead of restarting the application crazily, exponential backoff restart will increase the time between restarts and reduce the pressure on our database or external providers… Very easy to use:

Set this function on the terminal command line with the option — exp backoff restart delay

$ pm2 start app.js --exp-backoff-restart-delay=100

In the configuration file, use exp_backoff_restart_delay property.

server.config.js

module.exports = {
  script: 'server.js',
  exp_backoff_restart_delay: 100
}

When the application crashes unexpectedly and the option — exp backoff restart delay is activated, we will be able to see the new application state waiting restart.

By running pm2 logs , we will also see an increase in restart latency:

PM2      | App [throw:0] will restart in 100ms
PM2      | App [throw:0] exited with code [1] via signal [SIGINT]
PM2      | App [throw:0] will restart in 150ms
PM2      | App [throw:0] exited with code [1] via signal [SIGINT]
PM2      | App [throw:0] will restart in 225ms

As you can see, the restart delay between restarts will increase exponentially until the maximum value between restarts is 15000 milliseconds.

When the application returns to stable mode (uptime does not exceed 30 seconds), the restart delay will automatically reset to 0 milliseconds.

For more information about pm2, refer to pm2 tutorial

Источник

Restart strategies

Restart at cron time

Restart on file change

Memory based restart strategy

Restart Delay

No Auto Restart

Skip Auto Restart For Specific Exit Codes

Exponential Backoff Restart Delay

Exponential Backoff Restart Delay

Fixed Restart Delay

Memory based reload strategy

0second Downtime Reload

No Auto Restart

Javascript

PM2 production process manager

Table of contents

Quick start

Features

Windows specifics

Deployment — ecosystem.json

Using PM2 programmatically (via API)

Specific

Knowledge

More

Quick start

Different ways to launch a process

Features

Considerations

Hot reload / 0s reload

Graceful reload

Displaying logs in real-time

Flushing logs

Reloading all logs

Configuring logs

CLI

JSON / Programmatic

CLI

JSON

Programmatic

Units

Startup Systems support

User permissions

Related commands

Watch & Restart

JSON app declaration

Options

Schema

Considerations

Getting started with deployment

Commands

Using file key for authenticating

Simple example

API health endpoint

Knowledge

JSON configuration

Install PM2 development

Node 0.10.x doesn’t free the script port when stopped in cluster_mode

User tips from issues

Contributors

Sponsors

Restart the application using a scheduled task

Automatically restart the application after file changes

Restart when application reaches memory threshold

Delayed restart

Prohibit automatic restart

Specifies an exit code that does not restart automatically

Exponential backoff restart delay