Apache has been around since 1995 and is the most important web technology. The majority of businesses nowadays run-on Apache servers. Different servers operate in different ways and have different features and functions. For simple debugging, several servers keep server logs. Understanding how the server works is essential.
All errors encountered by the server while receiving or processing requests are recorded in the Apache error logs. These logs are accessible to admin users, who can troubleshoot them for immediate solutions.
These logs will provide you with useful information about the traffic to your application, errors that occur, and performance. This article will cover Apache web server logs, including log levels, format, and formatting as JSON.
We will cover the following:
- What are Apache Logs?
- What is the Apache Error Log?
- Apache Error Log vs Apache Access Log
- Where Do You Find the Apache Error Log?
- How Do You Check the Apache Error Log?
- Configuring Apache Logs
- Apache Error Log Levels
- Apache Error Log Format
- Apache Request Tracking
- Managed Dedicated Servers
- Unmanaged Dedicated Servers
- Assigning Nicknames
- Formatting as JSON
- How Do You Clear the Apache Error Log?
What are Apache Logs?
The Apache server keeps a record of all server actions in a text file known as Apache logs. These logs provide useful information about what resources are used and released. Additionally, the logs contain information about who visited those resources and for how long and all related data.
Not only that, but you’ll get all of the details on all of the errors that occurred. The admins can use these logged errors to look into fixing the errors and determining the root cause to eliminate the possibility of the error occurring again.
There isn’t a single Apache logging process. It involves several steps, including saving the logs in a specified area for future reference, analyzing and parsing the logs to extract the relevant information, and producing the graph for a better visual representation. There isn’t a single log type that Apache keeps track of. However, we’ll be concentrating on Apache error logs.
Types of Apache Log
There are two types of logs produced by Apache: Access logs and Error logs.
- Access Log
The access log keeps track of the requests that come into the webserver. This data could include what pages visitors are looking at, the status of requests, and how long it took the server to respond. - Error Log
The error log records any errors encountered by the web server when processing requests, such as missing files. It also contains server-specific diagnostic information.
What is the Apache Error Log?
If you can’t figure out what’s wrong with your server and it keeps throwing errors, it will become unreliable. We must keep information about the errors affecting the server’s performance for easy and seamless server operation.
As a result, Apache keeps error logs in a specific location. For simple troubleshooting, the server administrator sets up these log files and other details such as file size, error time, type of error, error message, and so on.
The Apache log keeps track of events handled by the Apache web server, such as requests from other computers, Apache responses, and internal Apache server operations.
These logs will continue to be created and stored at that location, making them accessible in the future. You must understand where these errors are saved, how to access them, and how to use the information they provide.
Apache Error Log vs. Apache Access Log
The next thing that people appear to be confused about is the difference between the access log and the error log. Are these two different concepts? Or do they refer to the same thing under different names?
They’re two very different things. The access log is not the same as the error log.
The access log keeps an account of all requests made to the web server, as well as who made them. You can think of it as a geekier version of an event guest log. It keeps track of information like visitor IP addresses, URL requests, responses, and so on.
While the access log may contain information about difficulties that arise, the error log—the subject of our discussion today—is dedicated to logging errors and problems that occur during the server’s operation. You can search for general information about web requests in the access log, but errors should be found in the error log.
Where do you find the Apache Error Log?
Several factors have a role in this:
- Apache is a cross-platform web server that can run on nearly any operating system. It’ll be difficult enough to figure out which version to download, let alone locate any log files it generates.
- The majority of Apache installations take place on Unix and Linux systems, which are even more fragmented in terms of where you can find stuff.
- Apache is highly customizable. While this gives it a lot of flexibility and power, it also makes items more difficult to find because you can put them everywhere.
So, what does it all add up to?
When you Google «where is the Apache error log,» Google could simply pop up a message saying, «No one knows where yours is.» This would save you a lot of clicking and bounce.
However, everything is not lost.
You could look up your operating system’s log file location or the default Apache log file location using Google. Keep in mind that you’ll have to search for your operating system. The ‘/var/log‘ directory is a common destination for log files on *nix systems.
Instead, we did investigate the location of your Apache configuration files. Because these are configurable Apache settings, you can identify the location of the error log (and other logs) once you have that.
For instance, consider the following example of log location configuration:
Update the below log path and log levels in the default configuration file — /etc/apache2/apache2.conf.
ErrorLog "/var/log/apache2/error.log"By the way, you’re not obligated to name the file «error.log.» Instead, any legal name you provide would suffice. If the specified file already exists, additional log entries will be appended to it.
You can also use a Syslog server to send log messages. Use the Syslog directive instead of a filename.
ErrorLog syslogThere are numerous options for the logging facility and the application name in the Syslog option. Details can be found here.
Finally, you can use a Linux command to redirect logs.
ErrorLog "|/usr/local/bin/httpd_errors"The pipe character tells Apache that the messages should be piped to the specified command.
Access logs for apache can be found in the following directory on the cPanel server.
/usr/local/apache/logs/access_logHow do you check the Apache Error Log?
How do you go about really checking it once you’ve located it?
You must first understand what it is to understand how it is.
A text file contains the Apache error log. So you can use any text editor or a command-line application like Cat or Tail to open it. These will offer you a quick preview of the contents and allow you to navigate through to find what you need. The entries in the file will be in reverse chronological order.
Let’s say you wish to utilize the «tail» command. In practice, how would you go about doing that?
Easy. To begin, start a shell session.
After that, you type the command:
tail -f /path-to/logYou’ll see the file’s last few entries after running the command above. You’ll also see new entries as they become available.
Since your log file contains errors, it should now be small. However, it could be large—too large to search quickly. In that situation, you might wish to use grep and regex to scan the contents of the error log. Alternatively, you may use a log aggregator to parse and turn the contents into data that you can readily query.
Configuring Apache Logs
The logging framework in Apache is highly flexible, allowing you to change logging behavior globally or for each virtual host. To adjust logging behavior, you can use several directives. The log level and log format directives are two of the most popular directives.
Apache Error Log Levels
It’s no surprise that the Apache error log, like any other log file, uses logging levels.
If you’re unfamiliar with the concept of logging levels, we recommend reading our post on the log levels.
In short, logging levels are labels that can be applied to log entries. The messages transmitted to the log file can then be filtered, thereby changing the verbosity of each message.
The LogLevel directive can be used to specify the desired level, as shown in the following example:
LogLevel warnThe default level is «warn,» although there are a variety of alternative options.
The levels are described as follows in the Apache documentation:
If you want to understand more about each level in-depth, go to Apache’s documentation.
Let’s look at how the two logging levels compare. We’ll compare the levels of warning and debug logs.
- Warning: You will only receive warning logs if the system is functioning normally but something does not appear to be correct. If these warnings are not addressed promptly, they may result in serious problems over time.
- Debug log level, on the other hand, logs practically everything that happens on the server. It might be errors or any other essential notification that you as a website administrator could find useful.
The following command in the main configuration file can be used to enable debug log level.
LogLevel alert rewrite:trace6Apache Error Log Format
You can also modify the format of log messages. We have been using the default format thus far. The ErrorLogFormat setting controls the message composition.
Let’s try a different setting to see what happens. We are going to utilize this example from the Apache documentation.
ErrorLogFormat "[%t] [%l] [pid %P] %F: %E: [client %a] %M"This is how a request appears right now:
[Thu Jun 09 11:46:18 2022] [fuga:error] [pid 6157:tid 4946] [client 144.110.67.126:20151] You can't copy the port without navigating the haptic SMTP card!
[Thu Jun 09 11:46:18 2022] [pariatur:info] [pid 5946:tid 7137] [client 35.8.11.82:2909] We need to transmit the primary SQL interface!Let’s go over how to use this format. The format string contains parameters that correspond to fields in a logging event.
The fields we used above are as follows:
For new connections and requests, you can define alternative formats. When a new client connects to the server, it is called a connection. A request is a communication that asks for something, such as a page or an image.
Apache uses connection and request formats when a new client connects or makes a web request. It records a message indicating that a client has established a connection or made a new request.
Apache Request Tracking
You can go a step further with logging messages for new connections and requests. Every new request or connection will generate a unique identifier in the Apache error log. This parameter can be used to group log entries. For identifiers, use the %L field.
Assigning Nicknames
You can give LogFormat strings nicknames, which you can use with the CustomLog directive to write logs in the format you specify. This allows you to use the same log format across multiple log files without having to change it each time. This is especially beneficial when many virtual hosts are utilizing separate log files.
Let’s say you want to build an example format called «vhost_combined.» Then, using the vhost_combined format, we’ll create a CustomLog directive that writes logs to a file.
LogFormat "%v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" vhost_combined
CustomLog /var/log/apache2/vhost.log vhost_combinedFormatting as JSON
If you save your logs as plain text, they’ll be easy to scan if you ever need to read them. This makes it difficult to understand your logs with tools like log management solutions, which require knowledge of how your logs are formatted. The default Apache log format is supported by most log management solutions, but if it isn’t, you should consider adopting a structured format like JSON.
JSON is a lightweight data storage format. JSON holds practically any data type and structure as a set of nestable name/value pairs. JSON is also self-documenting because the key name describes the data it contains. Strings, numbers, booleans, arrays, and null values are among the basic data types supported by JSON.
A LogFormat that saves logs in JSON format looks like this:
LogFormat "{ "time":"%t", "remoteIP":"%a", "host":"%V", "request":"%U", "query":"%q", "method":"%m", "status":"%>s", "userAgent":"%{User-agent}i", "referer":"%{Referer}i" }"Read must-know tips for JSON logging.
How Do You Clear the Apache Error Log?
Simply put, the log file will get quite large throughout a long period. You won’t need error messages from two years ago at some point. So you’d like to get rid of it to free up some disk space, either by truncating it or backing it up somewhere else.
When it comes to deleting the logs, there’s an interesting problem to solve. The server keeps a handle on the file for writing as long as it is operating. You can’t just open it and start editing it, or back it up and delete it.
Having root access and doing something like this—overwriting the file contents with nothing—might be the simplest method. However, we would recommend making a backup of the file first.
Alternatively, you can follow the instructions for log rotation and piped logs on Apache’s website. These solutions are more complicated, but they are more «proper» and less hacky.
However, regardless of how you approach deleting the logs or logging in general, our advice is to always err on the side of keeping as much information as possible.
Conclusion
The Apache error logs may appear to be simple. More than grepping log files or other ad hoc approaches are required to achieve uptime SLA. You should create a system for storing and analyzing Apache error logs. Tracking and analyzing HTTP error codes, finding outlier IP addresses, listing pages with slow response times, and other metrics that might improve uptime and MTTR.
Atatus
Logs Monitoring and Management
Atatus offers a
Logs Monitoring
solution which is delivered as a fully managed cloud service with minimal setup at any scale that requires no maintenance. It monitors logs from all of your systems and applications into a centralized and easy-to-navigate user interface, allowing you to troubleshoot faster.
We give a cost-effective, scalable method to centralized logging, so you can obtain total insight across your complex architecture. To cut through the noise and focus on the key events that matter, you can search the logs by hostname, service, source, messages, and more. When you can correlate log events with
APM
slow traces and errors, troubleshooting becomes easy.
Try your 14-day free trial of Atatus.
By Steve Smith
ASP.NET Core has built-in support for logging, and allows developers to easily leverage their preferred logging framework’s functionality as well. Implementing logging in your application requires a minimal amount of setup code. Once this is in place, logging can be added wherever it is desired.
Sections:
- Implementing Logging in your Application
- Configuring Logging in your Application
- Logging Recommendations
- Summary
View or download sample code
Implementing Logging in your Application¶
Adding logging to a component in your application is done by requesting either an ILoggerFactory or an ILogger<T> via Dependency Injection. If an ILoggerFactory is requested, a logger must be created using its CreateLogger method. The following example shows how to do this:
var logger = loggerFactory.CreateLogger("Catchall Endpoint"); logger.LogInformation("No endpoint found for request {path}", context.Request.Path);
When a logger is created, a category name must be provided. The category name specifies the source of the logging events. By convention this string is hierarchical, with categories separated by dot (.) characters. Some logging providers have filtering support that leverages this convention, making it easier to locate logging output of interest. In this article’s sample application, logging is configured to use the built-in ConsoleLogger (see Configuring Logging in your Application below). To see the console logger in action, run the sample application using the dotnet run command, and make a request to configured URL (localhost:5000). You should see output similar to the following:
You may see more than one log statement per web request you make in your browser, since most browsers will make multiple requests (i.e. for the favicon file) when attempting to load a page. Note that the console logger displayed the log level (info in the image above) followed by the category ([Catchall Endpoint]), and then the message that was logged.
The call to the log method can utilize a format string with named placeholders (like {path}). These placeholders are populated in the order in which they appear by the args values passed into the method call. Some logging providers will store these names along with their mapped values in a dictionary that can later be queried. In the example below, the request path is passed in as a named placeholder:
logger.LogInformation("No endpoint found for request {path}", context.Request.Path);
In your real world applications, you will want to add logging based on application-level, not framework-level, events. For instance, if you have created a Web API application for managing To-Do Items (see Building Your First Web API with ASP.NET Core MVC and Visual Studio), you might add logging around the various operations that can be performed on these items.
The logic for the API is contained within the TodoController, which uses Dependency Injection to request the services it requires via its constructor. Ideally, classes should follow this example and use their constructor to define their dependencies explicitly as parameters. Rather than requesting an ILoggerFactory and creating an instance of ILogger explicitly, TodoController demonstrates another way to work with loggers in your application — you can request an ILogger<T> (where T is the class requesting the logger).
[Route("api/[controller]")] public class TodoController : Controller { private readonly ITodoRepository _todoRepository; private readonly ILogger<TodoController> _logger; public TodoController(ITodoRepository todoRepository, ILogger<TodoController> logger) { _todoRepository = todoRepository; _logger = logger; } [HttpGet] public IEnumerable<TodoItem> GetAll() { _logger.LogInformation(LoggingEvents.LIST_ITEMS, "Listing all items"); EnsureItems(); return _todoRepository.GetAll(); }
Within each controller action, logging is done through the use of the local field, _logger, as shown on line 17, above. This technique is not limited to controllers, but can be utilized by any of your application services that utilize Dependency Injection.
Working with ILogger<T>¶
As we have just seen, your application can request an instance of ILogger<T> as a dependency in a class’s constructor, where T is the type performing logging. The TodoController shows an example of this approach. When this technique is used, the logger will automatically use the type’s name as its category name. By requesting an instance of ILogger<T>, your class doesn’t need to create an instance of a logger via ILoggerFactory. You can use this approach anywhere you don’t need the additional functionality offered by ILoggerFactory.
Logging Verbosity Levels¶
When adding logging statements to your application, you must specify a LogLevel. The LogLevel allows you to control the verbosity of the logging output from your application, as well as the ability to pipe different kinds of log messages to different loggers. For example, you may wish to log debug messages to a local file, but log errors to the machine’s event log or a database.
ASP.NET Core defines six levels of logging verbosity, ordered by increasing importance or severity:
- Trace
- Used for the most detailed log messages, typically only valuable to a developer debugging an issue. These messages may contain sensitive application data and so should not be enabled in a production environment. Disabled by default. Example:
Credentials: {"User":"someuser", "Password":"P@ssword"} - Debug
- These messages have short-term usefulness during development. They contain information that may be useful for debugging, but have no long-term value. This is the default most verbose level of logging. Example:
Entering method Configure with flag set to true - Information
- These messages are used to track the general flow of the application. These logs should have some long term value, as opposed to
Verboselevel messages, which do not. Example:Request received for path /foo - Warning
- The Warning level should be used for abnormal or unexpected events in the application flow. These may include errors or other conditions that do not cause the application to stop, but which may need to be investigated in the future. Handled exceptions are a common place to use the Warning log level. Examples:
Login failed for IP 127.0.0.1orFileNotFoundException for file foo.txt - Error
- An error should be logged when the current flow of the application must stop due to some failure, such as an exception that cannot be handled or recovered from. These messages should indicate a failure in the current activity or operation (such as the current HTTP request), not an application-wide failure. Example:
Cannot insert record due to duplicate key violation - Critical
- A critical log level should be reserved for unrecoverable application or system crashes, or catastrophic failure that requires immediate attention. Examples: data loss scenarios, out of disk space
The Logging package provides helper extension methods for each LogLevel value, allowing you to call, for example, LogInformation, rather than the more verbose Log(LogLevel.Information, ...) method. Each of the LogLevel-specific extension methods has several overloads, allowing you to pass in some or all of the following parameters:
- string data
- The message to log.
- EventId eventId
- A numeric id to associate with the log, which can be used to associate a series of logged events with one another. Event IDs should be static and specific to a particular kind of event that is being logged. For instance, you might associate adding an item to a shopping cart as event id 1000 and completing a purchase as event id 1001. This allows intelligent filtering and processing of log statements.
- string format
- A format string for the log message.
- object[] args
- An array of objects to format.
- Exception error
- An exception instance to log.
Note
The EventId type can be implicitly casted to int, so you can just pass an int to this argument.
Note
Some loggers, such as the built-in ConsoleLogger used in this article, will ignore the eventId parameter. If you need to display it, you can include it in the message string. This is done in the following sample so you can easily see the eventId associated with each message, but in practice you would not typically include it in the log message.
In the TodoController example, event id constants are defined for each event, and log statements are configured at the appropriate verbosity level based on the success of the operation. In this case, successful operations log as Information and not found results are logged as Warning (error handling is not shown).
[HttpGet] public IEnumerable<TodoItem> GetAll() { _logger.LogInformation(LoggingEvents.LIST_ITEMS, "Listing all items"); EnsureItems(); return _todoRepository.GetAll(); } [HttpGet("{id}", Name = "GetTodo")] public IActionResult GetById(string id) { _logger.LogInformation(LoggingEvents.GET_ITEM, "Getting item {0}", id); var item = _todoRepository.Find(id); if (item == null) { _logger.LogWarning(LoggingEvents.GET_ITEM_NOTFOUND, "GetById({0}) NOT FOUND", id); return NotFound(); } return new ObjectResult(item); }
Note
It is recommended that you perform application logging at the level of your application and its APIs, not at the level of the framework. The framework already has logging built in which can be enabled simply by setting the appropriate logging verbosity level.
To see more detailed logging at the framework level, you can adjust the LogLevel specified to your logging provider to something more verbose (like Debug or Trace). For example, if you modify the AddConsole call in the Configure method to use LogLevel.Trace and run the application, the result shows much more framework-level detail about each request:
The console logger prefixes debug output with “dbug: ”; there is no trace level debugging enabled by the framework by default. Each log level has a corresponding four character prefix that is used, so that log messages are consistently aligned.
| Log Level | Prefix |
|---|---|
| Critical | crit |
| Error | fail |
| Warning | warn |
| Information | info |
| Debug | dbug |
| Trace | trce |
Scopes¶
In the course of logging information within your application, you can group a set of logical operations within a scope. A scope is an IDisposable type returned by calling the ILogger.BeginScope<TState> method, which lasts from the moment it is created until it is disposed. The built-in TraceSource logger returns a scope instance that is responsible for starting and stopping tracing operations. Any logging state, such as a transaction id, is attached to the scope when it is created.
Scopes are not required, and should be used sparingly, if at all. They’re best used for operations that have a distinct beginning and end, such as a transaction involving multiple resources.
Configuring Logging in your Application¶
To configure logging in your ASP.NET Core application, you should resolve ILoggerFactory in the Configure method of your Startup class. ASP.NET Core will automatically provide an instance of ILoggerFactory using Dependency Injection when you add a parameter of this type to the Configure method.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
Once you’ve added ILoggerFactory as a parameter, you configure loggers within the Configure method by calling methods (or extension methods) on the logger factory. We have already seen an example of this configuration at the beginning of this article, when we added console logging by calling loggerFactory.AddConsole.
Note
You can optionally configure logging when setting up Hosting, rather than in Startup.
Each logger provides its own set of extension methods to ILoggerFactory. The console, debug, and event log loggers allow you to specify the minimum logging level at which those loggers should write log messages. The console and debug loggers provide extension methods accepting a function to filter log messages according to their logging level and/or category (for example, logLevel => logLevel >= LogLevel.Warning or (category, loglevel) => category.Contains("MyController") && loglevel >= LogLevel.Trace). The event log logger provides a similar overload that takes an EventLogSettings instance as argument, which may contain a filtering function in its Filter property. The TraceSource logger does not provide any of those overloads, since its logging level and other parameters are based on the SourceSwitch and TraceListener it uses.
A LoggerFactory instance can optionally be configured with custom FilterLoggerSettings. The example below configures custom log levels for different scopes, limiting system and Microsoft built-in logging to warnings while allowing the app to log at debug level by default. The WithFilter method returns a new ILoggerFactory that will filter the log messages passed to all logger providers registered with it. It does not affect any other ILoggerFactory instances, including the original ILoggerFactory instance.
loggerFactory .WithFilter(new FilterLoggerSettings { { "Microsoft", LogLevel.Warning }, { "System", LogLevel.Warning }, { "ToDoApi", LogLevel.Debug } }) .AddConsole();
Configuring TraceSource Logging¶
When running on the full .NET Framework you can configuring logging to use the existing System.Diagnostics.TraceSource libraries and providers, including easy access to the Windows event log. TraceSource allows you to route messages to a variety of listeners and is already in use by many organizations.
First, be sure to add the Microsoft.Extensions.Logging.TraceSource package to your project (in project.json), along with any specific trace source packages you’ll be using (in this case, TextWriterTraceListener):
"Microsoft.AspNetCore.Mvc": "1.0.0", "Microsoft.AspNetCore.Server.Kestrel": "1.0.0", "Microsoft.AspNetCore.Server.IISIntegration": "1.0.0", "Microsoft.AspNetCore.StaticFiles": "1.0.0", "Microsoft.Extensions.Logging": "1.0.0", "Microsoft.Extensions.Logging.Console": "1.0.0", "Microsoft.Extensions.Logging.Filter": "1.0.0", "Microsoft.Extensions.Logging.TraceSource": "1.0.0" }, "tools": { "Microsoft.AspNetCore.Server.IISIntegration.Tools": {
The following example demonstrates how to configure a TraceSourceLogger instance for an application, logging only Warning or higher priority messages. Each call to AddTraceSource takes a TraceListener. The call configures a TextWriterTraceListener to write to the console window. This log output will be in addition to the console logger that was already added to this sample, but its behavior is slightly different.
// add Trace Source logging var testSwitch = new SourceSwitch("sourceSwitch", "Logging Sample"); testSwitch.Level = SourceLevels.Warning; loggerFactory.AddTraceSource(testSwitch, new TextWriterTraceListener(writer: Console.Out));
The sourceSwitch is configured to use SourceLevels.Warning, so only Warning (or higher) log messages are picked up by the TraceListener instance.
The API action below logs a warning when the specified id is not found:
[HttpGet("{id}", Name = "GetTodo")] public IActionResult GetById(string id) { _logger.LogInformation(LoggingEvents.GET_ITEM, "Getting item {0}", id); var item = _todoRepository.Find(id); if (item == null) { _logger.LogWarning(LoggingEvents.GET_ITEM_NOTFOUND, "GetById({0}) NOT FOUND", id); return NotFound(); } return new ObjectResult(item); }
To test out this code, you can trigger logging a warning by running the app from the console and navigating to http://localhost:5000/api/Todo/0. You should see output similar to the following:
The yellow line with the “warn: ” prefix, along with the following line, is output by the ConsoleLogger. The next line, beginning with “TodoApi.Controllers.TodoController”, is output from the TraceSource logger. There are many other TraceSource listeners available, and the TextWriterTraceListener can be configured to use any TextWriter instance, making this a very flexible option for logging.
Configuring Other Providers¶
In addition to the built-in loggers, you can configure logging to use other providers. Add the appropriate package to your project.json file, and then configure it just like any other provider. Typically, these packages include extension methods on ILoggerFactory to make it easy to add them.
- elmah.io — provider for the elmah.io service
- Loggr — provider for the Loggr service
- NLog — provider for the NLog library
- Serilog — provider for the Serilog library
You can create your own custom providers as well, to support other logging frameworks or your own internal logging requirements.
Logging Recommendations¶
The following are some recommendations you may find helpful when implementing logging in your ASP.NET Core applications.
- Log using the correct
LogLevel. This will allow you to consume and route logging output appropriately based on the importance of the messages. - Log information that will enable errors to be identified quickly. Avoid logging irrelevant or redundant information.
- Keep log messages concise without sacrificing important information.
- Although loggers will not log if disabled, consider adding code guards around logging methods to prevent extra method calls and log message setup overhead, especially within loops and performance critical methods.
- Name your loggers with a distinct prefix so they can easily be filtered or disabled. Remember the
Create<T>extension will create loggers named with the full name of the class. - Use Scopes sparingly, and only for actions with a bounded start and end. For example, the framework provides a scope around MVC actions. Avoid nesting many scopes within one another.
- Application logging code should be related to the business concerns of the application. Increase the logging verbosity to reveal additional framework-related concerns, rather than implementing yourself.
Summary¶
ASP.NET Core provides built-in support for logging, which can easily be configured within the Startup class and used throughout the application. Logging verbosity can be configured globally and per logging provider to ensure actionable information is logged appropriately. Built-in providers for console and trace source logging are included in the framework; other logging frameworks can easily be configured as well.
Today features another post about the nuts and bolts of logging. This time, I’ll be talking about the Apache error log in some detail.
Originally, I had a different plan and outline for this post. But then I started googling for good reference material.
And what I found were way more questions than answers about the topic, many on various Stack Exchange sites. It seems that information about the Apache error log is so scarce that people can’t agree on where to ask questions, let alone get answers to them.
So let’s change that. I’m going to phrase this as a Q&A-based outline, hopefully answering all of the questions you might have come looking for—if you googled the term—while also providing a broad narrative for regular readers.
First of All, What Is the Apache Error Log?
First things first. What exactly is this thing we’re talking about here? To understand that, you first need to understand what Apache is. Apache is a web server, in the software sense of the term. As briefly as possible, this means that it’s the application in which you put your website’s files, and it manages inbound HTTP requests.
Now as you might imagine, something that manages the websites you build is a fairly involved and fairly important piece of software. And, as such, it produces a lot of log data.
Apache’s error log is part of that log data that it produces. It’s a standalone file that specifically contains information about errors produced as the Apache web server starts up and runs.
According to Apache’s section on the subject, it is “the most important log file.”
Is the Apache Error Log a Different Thing from the Apache Access Log?
So there you have it. The Apache error log is a log file (and the most important log file) of Apache’s runtime errors.
The next thing people seem to wonder about a lot has to do with differentiating between the access log and the error log. Are these two different things? Or is it just kind of two names for the same thing?
They’re two different, distinct things. The error log is not the same thing as the access log.
For those who follow the blog, you may recall that I’ve posted in detail about the Apache access log. The access log keeps track of all of the requests that come into the web server and who has sent them. You can think of it as the really geeky equivalent of an event guest log. It keeps track of things like IP addresses of visitors, the URL requested, the response, etc.
Now, while the access log may have information related to problems that happen, the error log—the subject of our focus today—is specifically dedicated to logging errors and problems that occur with the running of the server. You can mine the access log for general information about web requests but look in the error log for, well, errors.
Where Do You Find the Apache Error Log?
The question about where you find the error log seems to cause absolutely the most confusion. The reason there’s so much confusion? Well, there are actually several contributing factors:
- Apache is a cross-platform web server that will run on just about any OS you might use. Just figuring out which version to download will confuse you, let alone finding some log file it produces.
- Most Apache installations occur on Unix and Linux system, which have even further fragmentation as to where you might find things.
- Apache is extremely configurable. And while that makes it extremely flexible and powerful, it also makes things even harder to find, since you can put them anywhere.
What does all of this add up to? Well, when you type “where is the Apache error log” into Google, Google could save you a lot of clicking and bouncing by just popping up a message saying, “No one actually knows where yours is.”
All is not lost, though.
You could google your operating system to see where it tends to keep log files or where default Apache log files generally go. Understand that you’ll have to search by your specific operating system. On *nix systems, a typical location for the log files would be under the ‘/var/log’ directory.
But what I’d do instead is actually figure out where your Apache config files are. Once you’ve got that, you can find the location of the error log (and other logs) because they’re configurable Apache settings.
For instance, this is a typical example of the log’s location configuration:
ErrorLog "/var/log/apache2/error.log"
By the way, you’re not forced to use “error.log” as the file name. Instead, any valid name you inform will do. If the name informed refers to an existent file, new log entries will be appended to it.
How Do You Check the Apache Error Log?
Once you’ve found it, how do you go about actually checking it? Well, to understand that, understand what it is.
The Apache error log is a text file. So you can open it with any text editor or with a command line utility like tail or cat. Those will give you a quick peek at the contents and let you scroll through to find what you’re looking for. The file’s entries will be in chronological order.
Suppose you want to use the “tail” utility. How would you go about that in practice? Easy. First, you open a shell session. You then type the following command:
tail ?f /path-to/log
After running the command above you’ll see the file’s last few entries. You’ll also continue seeing new entries, as they occur.
Now, hopefully, your log file is small since it contains errors. But it might be big—too big to search easily. In that case, you might want to search the error log’s contents using grep and regex. Or you might consider employing a log aggregator to parse and turn the contents into data that you can query easily.
What Are the Apache Error Log Levels?
Since the Apache error log is a log file, not that different in concept from any other log file, it doesn’t come as a surprise that it also makes use of logging levels.
If you aren’t familiar with the concept of logging levels, we invite you to read the post we’ve published about it.
In a nutshell, logging levels are labels that can be applied to log entries. Then can then be used to filter the messages sent to the log file, effectively adjusting the verbosity in each message.
You can specify the desired level using the LogLevel directive, like in the following example:
LogLevel warn
The default level is indeed “warn”, but there are many other levels available:
Level, emerg , alert , crit , error , warn , notice, info , debug , trace1, trace2, trace3, trace4, trace5, trace6, trace7, trace8
You can head to Apache’s documentation if you want to learn about each level in detail.
Let’s talk try to compare the two logging levels here. We will compare warning and debug log levels. Warning, you only receive warning logs when the system is working as expected but something doesn’t seem right. If these warnings are not attended to on time, they might bring some real errors as time go on. On the other hand, debug log level logs almost everything that is happening on the server. It can be errors and just any other important message that can be helpful to you as a website administrator.
You can enable debug log level writing the flowing command in the main configuration file.
LogLevel alert rewrite:trace6
How to Use Grep to Get Information from the Apache Error Log?
As we’ve just mentioned, it’s possible to use the grep command to search the contents of your Apache error log file. So, let’s see some quick examples of how that can be done.
Suppose you want to filter your log entries with the warn level. This is the command you should run, based on the example location for an Apache error log mentioned a few sections back:
grep warn /var/log/apache2/error.log
Similarly, if you wanted to retrieve log entries marked with the notice level, you’d just have to replace warn with notice in the command above. Easy peasy.
Keep in mind, though, that Apache logs—the error log included—tend to be extremely long. Let’s say you only wanted to output the last 15 lines of your log. You can use the command tail to accomplish that.
Tail is a command that outputs the tail—that is, the end—of a file. By default, it outputs the last 10 lines. You can specify the number of lines using the -n option:
tail -n 5 /var/log/apache2/error.log
You can use tail along with grep in order to get a subset of log entries, then filter them. That’s what we do in the following example: we get five lines from the log file, then feed them to the grep command, which filters for entries containing “warn.’
tail -n 5 /var/log/apache2/error.log | grep warn
Now, suppose you want to do negative filtering. You want to see all log entries except those with the warn level. Can you do it using the grep command? You bet you can! You’d only have to add the -v option to the grep command:
grep -v warn /var/log/apache2/error.log
How You Should View the Apache Error Log—The ErrorLogFormat Directive
That covers how you can get at the file’s contents in the broadest sense. But how do you actually view and interpret the thing, semantically? What does each entry actually mean?
Well, let’s consider an example, taken from this page. This is representative of what an error log file entry could look like:
#Simple example
ErrorLogFormat "[%t] [%l] [pid %P] %F: %E: [client %a] %M"
#And the output:
[Fri Dec 16 01:46:23 2005] [error] [client 1.2.3.4] Directory index forbidden by rule: /home/test/ [Fri Dec 16 01:54:34 2005] [error] [client 1.2.3.4] Directory index forbidden by rule: /apache/web-data/test2 [Fri Dec 16 02:25:55 2005] [error] [client 1.2.3.4] Client sent malformed Host header [Mon Dec 19 23:02:01 2005] [error] [client 1.2.3.4] user test: authentication failure for "/~dcid/test1": Password Mismatch
You can probably infer the format here of each entry: timestamp, log level, remote host, error message.
Unlike the access log that I mentioned earlier, you don’t have endless customization options here. The format of this particular file is actually fixed. But entries in this file will also correspond to entries in the access log, which you can configure.
You can also create custom log files of your choosing.
Managed Dedicated Servers
If you are using a managed dedicated server, there are a number of directories where you can find the Apache error logs. On cPanel server, you can find the apache access_logs in the following directory.
/usr/local/apache/logs/access_log
Now the access_log is where all the http requests are logged to.
Apache errors can be found in this directory /usr/local/apache/logs/error_log. All the Apache errors are logged in the error_log.
Another important folder you can look for is the demolog folder. Demologs are logs of domains, they are generated from the requests made to the domain. The directory can look like the following
/usr/local/apache/domlogs
Unmanaged Dedicated Servers
Like the name suggests, this is a type of server that is not managed by the hosting company and having that said, you have full control of this server and can decide where the Apache server can be writing the logs. The directories and folders will be up to your tech team.
How Do You Clear the Apache Error Log?
I’ll bookend this post by answering a question that seems logical to wrap up. We’ve looked at what the error log is, where you find it, how you view it, and how you interpret it. How do you clear it? And why would you want to?
Well, let’s answer that second question first.
Simply put, over the course of a lot of time, that log file is going to get pretty unwieldy. And at some point, you’re not going to need error messages from two years ago anymore. So you want to clear it out to clean up some disk space, either truncating it outright or backing up a copy somewhere else.
As for clearing the logs themselves, there’s an interesting issue to contend with there. As long as the server is running, it holds a handle to the file for writing. So you can’t just open it and start editing it, and you can’t just back it up and delete it.
Perhaps the simplest technique is to have root access and do something like this—overwriting the file contents with nothing. I’d suggest making a copy of the file first, though.
Alternatively, you can follow the instructions here, on Apache’s site, related to log rotation and piped logs. These are more involved but more “proper” and less hack-y solutions.
But however you approach clearing the logs or logging in general, my recommendation would be always to err on the side of preserving as much information as you possibly can.
Apache Error Log: No Longer a Stranger
Today’s post was another installment of our series that covers different aspects of logging. Thematically, this post doesn’t differ dramatically from the previous ones. After all, it covers the Apache error log, which is a special kind of log file. Structurally, though, this post was a novelty, since we’ve adopted a Q&A format for it.
Throughout the post, we’ve walked you through a list of common questions about the Apache error log. You’ve learned, among other things:
- what the Apache error log is
- where can you find it
- how to check the log
- the best way to visualize the log’s content
By now, you should have a solid grasp of the Apache error log fundamentals.
Hopefully, you can apply the knowledge you’ve obtained from this post to troubleshoot issues quickly, accessing the precious data contained on your apache log files and turning them into valuable insights for your organization.
Learn more about Scalyr by checking out a demo here.
Apache — это кроссплатформенный HTTP-сервер с открытым исходным кодом. Он имеет множество мощных функций, которые можно расширить с помощью самых разных модулей. При управлении веб-серверами Apache одной из наиболее частых задач, которые вы будете выполнять, является проверка файлов журнала.
Знание того, как настраивать и читать журналы, очень полезно при устранении неполадок сервера или приложений, поскольку они предоставляют подробную информацию об отладке.
Apache записывает свои события в журналы двух типов: журналы доступа и журналы ошибок. Журналы доступа включают информацию о клиентских запросах, а журналы ошибок — информацию о проблемах сервера и приложений.
В этой статье описывается, как настроить и прочитать журналы доступа и ошибок Apache.
Настройка журнала доступа
Веб-сервер Apache генерирует новое событие в журнале доступа для всех обработанных запросов. Каждая запись события содержит отметку времени и включает различную информацию о клиенте и запрошенном ресурсе. Журналы доступа показывают местоположение посетителей, страницу, которую они посещают, сколько времени они проводят на странице и многое другое.
Директива CustomLog определяет расположение файла журнала и формат регистрируемых сообщений.
Базовый синтаксис директивы CustomLog следующий:
CustomLog log_file format [condition];
log_file может относиться либо к ServerRoot либо к полному пути к файлу журнала. Сообщения журнала также можно передать другой программе с помощью символа вертикальной черты | .
Второй аргумент, format определяет формат сообщений журнала. Это может быть явное определение формата или псевдоним, определяемый директивой LogFormat .
LogFormat "%h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" combined
CustomLog logs/access.log combined
CustomLog logs/access.log "%h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i""
Чтобы не повторять один и тот же код несколько раз, предпочтительнее определить директиву LogFormat и использовать ее в качестве псевдонима в директиве CustomLog .
Полный список всех строк формата и модификаторов см. В документации модуля «mod_log_config» .
Третий аргумент [condition] является необязательным и позволяет записывать сообщения журнала только при выполнении определенного условия. Обычно это делается с использованием переменных окружения. Условие может быть отменено с помощью ! условное обозначение.
Например, если вы хотите исключить запросы к файлам css для записи в файл журнала, вы должны использовать следующее:
SetEnvIf Request_URI .css$ css-file
CustomLog logs/access.log custom env=!css-file
Чтобы изменить формат ведения журнала, вы можете определить новую директиву LogFormat или переопределить формат по умолчанию. Обычно лучше определить новый формат.
Хотя журнал доступа предоставляет очень полезную информацию, он занимает дисковое пространство и может повлиять на производительность сервера. Если на вашем сервере мало ресурсов и у вас загруженный веб-сайт, вы можете отключить журнал доступа.
Для этого просто закомментируйте или удалите директиву CustomLog из разделов конфигурации основного сервера и виртуального сервера.
Если вы хотите отключить журнал доступа только для одного виртуального хоста, установите для первого аргумента директивы CustomLog значение /dev/null :
CustomLog /dev/null combined
Настройка журнала ошибок
Apache записывает сообщения об ошибках приложения и общих ошибках сервера в файл журнала ошибок. Если вы испытываете ошибки в своем веб-приложении, журнал ошибок — это первое место, с которого можно начать поиск и устранение неисправностей.
Директива ErrorLog определяет расположение имени журнала ошибок. Он имеет следующий вид:
Если путь к log_file не является абсолютным, он устанавливается относительно ServerRoot . Сообщения об ошибках также могут быть переданы другой программе с помощью символа вертикальной черты | .
Параметр LogLevel устанавливает уровень ведения журнала. Ниже перечислены уровни в порядке их серьезности (от низкого до высокого):
trace1—trace8— сообщения трассировки.-
debug—debugсообщения. -
info— Информационные сообщения. -
notice— Уведомления. -
warn— Предупреждения. -
error— Ошибки при обработке запроса. -
crit— Критические проблемы. Требуется быстрое действие. -
alert— Оповещения. Действия должны быть предприняты немедленно. -
emerg— Чрезвычайная ситуация. Система находится в непригодном для использования состоянии.
Каждый уровень журнала включает в себя более высокие уровни. Например, если вы установите уровень журнала , чтобы warn , Apache также записывает error , crit , alert и emerg сообщения.
Если параметр LogLevel не указан, по умолчанию используется warn . Рекомендуется установить уровень как минимум crit .
Директива ErrorLogFormat определяет формат журнала ошибок. В большинстве дистрибутивов Linux сервер Apache использует формат по умолчанию, которого достаточно в большинстве случаев.
Виртуальные хосты и глобальное ведение журнала
Поведение журнала и расположение файлов можно установить глобально или для каждого виртуального хоста.
Затем в контексте основного сервера ErrorLog директивы CustomLog или ErrorLog , сервер записывает все сообщения журнала в те же файлы журнала доступа и ошибок. В противном случае, если директивы помещаются в блок <VirtualHost> , в указанный файл записываются только сообщения журнала для этого виртуального хоста.
Директива журнала, установленная в блоке <VirtualHost> заменяет директиву, установленную в контексте сервера.
Виртуальные хосты без директив CustomLog или ErrorLog будут записывать свои сообщения журнала в глобальные журналы сервера.
Для лучшей читаемости рекомендуется установить отдельные файлы журнала доступа и ошибок для каждого виртуального хоста. Вот пример:
<VirtualHost *:80>
ServerName example.com
ServerAlias www.example.com
ServerAdmin [email protected]
DocumentRoot /var/www/example.com/public
LogLevel warn
ErrorLog /var/www/example.com/logs/error.log
CustomLog /var/www/example.com/logs/access.log combined
</VirtualHost>
Всякий раз, когда вы изменяете файл конфигурации, вам необходимо перезапустить службу Apache, чтобы изменения вступили в силу.
Расположение файлов журнала
По умолчанию в дистрибутивах на основе Debian, таких как Ubuntu , журналы доступа и ошибок расположены в каталоге /var/log/apache2 . В CentOS файлы журнала помещаются в каталог /var/log/httpd .
Чтение и понимание файлов журнала Apache
Файлы журнала можно открывать и анализировать с помощью стандартных команд, таких как cat , less , grep , cut , awk и т. Д.
Вот пример записи из файла журнала доступа, в котором используется формат журнала combine Debian:
192.168.33.1 - - [08/Jan/2020:21:39:03 +0000] "GET / HTTP/1.1" 200 6169 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
Давайте разберемся, что означает каждое поле записи:
%h—192.168.33.1— Имя хоста или IP-адрес клиента, выполняющего запрос.-
%l—-— Имя удаленного журнала. Если имя пользователя не задано, в этом поле отображается-. -
%u—-— Если запрос аутентифицирован, отображается имя удаленного пользователя. -
%t—[08/Jan/2020:21:39:03 +0000]— Время локального сервера. -
"%r"—"GET / HTTP/1.1"— первая строка запроса. Тип запроса, путь и протокол. -
%>s—200— окончательный код ответа сервера. Если символ>не используется и запрос был перенаправлен внутри, он покажет статус исходного запроса. -
%O—396— Размер ответа сервера в байтах. -
"%{Referer}i"—"-"— URL перехода. -
"%{User-Agent}i"—Mozilla/5.0 ...— Пользовательский агент клиента (веб-браузера).
Используйте команду tail для просмотра файла журнала в режиме реального времени:
tail -f access.log Выводы
Файлы журналов содержат полезную информацию о проблемах с сервером и о том, как посетители взаимодействуют с вашим сайтом.
Apache имеет очень настраиваемую систему ведения журнала, которая позволяет настраивать журналы доступа и ошибок в соответствии с вашими потребностями.
Если у вас есть какие-либо вопросы или отзывы, не стесняйтесь оставлять комментарии.