Redirect NGINX and Rails Logs to STDOUT and STDERR

[rhuanbarreto]

Is it possible to redirect the logs from NGINX and Rails to STDOUT and STDERR?

Configuring Rails to output to STDOUT don't work.

[rhuanbarreto]

I can see that the nginx logs are forwarded by the nginx log forwarder process. But I can't find this here to implement a rails log forwarder.

[iconara]

@rhuanbarreto I had a similar problem and solved it by making Rails log to STDOUT, that made its logs end up in /var/log/nginx/error.log, which is what ends up in the Docker output. Does this not work for you? Perhaps #224 could shed some more light in that case.

[rhuanbarreto]

Anyone from Phusion to help?

[thomasbalsloev]

OK, I got it.

In your Rails env config make sure your logging output is sent to STDOUT.

However, this wont make the output go directly to STDOUT as stated in the Phusion Passenger docs: https://www.phusionpassenger.com/library/admin/nginx/log_file/
The output will be captured by the passenger logger and redirected to whatever is set in "passenger_log_file" setting which defaults to the Nginx error log file.

Note also that the passenger log level ("passenger_log_level" setting) will override your Rails log level. I.e. if your Rails log level is 4 and your passenger level is 2 you wont get what you thought you would get. That configuration would only get you "warn" and "error" logs from Rails.

Finally you can stream the logs from whatever log file you have chosen to stdout by:
ln -sf /dev/stdout /var/log/nginx/error.log

If you point your nginx access log and error log to the same file and set "passenger_log_file" to this path you can get all logs in your docker output.

[iconara]

We ended up simplifying our setup even further, like this:

# /etc/nginx/main.d/env.conf

env RAILS_LOG_TO_STDOUT;

# /etc/nginx/sites-enabled/app.conf

passenger_log_file /dev/stdout;

server {
  # …
  access_log /dev/stdout;
  error_log /dev/stdout;
}

With this config we're able to get all logs to stdout, and then use Docker to ship them to CloudWatch Logs.

[chewi]

@iconara's solution works.

It wasn't mentioned above but the current logging setup is broken. Whether you send Rails to STDOUT or not, Passenger's output is lost as soon as log rotation occurs. This is triggered by cron.daily. Passenger doesn't notice and continues writing to the old file (e.g. /var/log/nginx/error.log.1) until you manually call passenger-config reopen-logs. /etc/logrotate.d/nginx should do this for you but it all seems pointless to me. Why not have nginx and Passenger just write to STDOUT in the first place as above? Or at least move the access_log and error_log directives into a small included config file so we can redefine the global values cleanly.

[CamJN]

Passenger standalone uses nginx internally and nginx has longstanding reliability issues with logging to stdout. That coupled with the fact that we use the phusion baseimage, which doesn't run nginx as pid 1 and therefore would require a log forwarder anyway is why we don't just log to stdout and stderr directly.

[chewi]

@CamJN, please could you point me to evidence of that unreliability? It sounds concerning but it's all we can do without a fix to the logrotate config.

[CamJN]

A good deal of it is just experience, however this issue being WONTFIX means I'm not going to set stdout as the default nginx log location for passenger-docker. Updating the logrotate config is much more palatable.

[chewi]

Thanks for that. They suggest stderr, which seems to work too. From Docker's perspective, it doesn't make any difference? Although you said this only happens with daemon on but this image has daemon off anyway so where's the problem? I feel like I'm missing something here!

[CamJN]

While I was mistaken about the applicability of that issue to how this image works by default, (the issue that led to it being opened was about using passenger standalone in docker), we can't really expect people to not change any configuration in the image, that's a core tenant of docker after all. And I don't want to replace one weird failure case with another. Adding a postrotate hook to reopen the passenger logs is a good solution IMO, that works well with the init system and existing design.

[rhuanbarreto]

I'm closing the issue because I don't think this will be solved by Passenger. The docker image is too fat to be used in production with swarms or kubernetes clusters. It's like configuring a server itself that drains 2GB RAM for simple applications.

My solution was run away from passenger as a web server and use the ruby base image with Puma in one container and NginX in another container as a proxy.