How do you define a log in python?
Vinay Sajip Show
Basic Logging Tutorial¶Logging is a means of tracking events that happen when some software runs. The software’s developer adds logging calls to their code to indicate that certain events have occurred. An event is described by a descriptive message which can optionally contain variable data (i.e. data that is potentially different for each occurrence of the event). Events also have an importance which the developer ascribes to the event; the importance can also be called the level or severity. When to use logging¶Logging provides a set of convenience functions for simple logging usage. These are
The logging functions are named after the level or severity of the events they are used to track. The standard levels and their applicability are described below (in increasing order of severity):
The default level is Events that are tracked can be handled in different ways. The simplest way of handling tracked events is to print them to the console. Another common way is to write them to a disk file. A simple example¶A very simple example is: import logging logging.warning('Watch out!') # will print a message to the console logging.info('I told you so') # will not print anything If you type these lines into a script and run it, you’ll see: printed out on the console. The Logging to a file¶A very common situation is that of recording logging events in a file, so let’s look at that next. Be sure to try the following in a newly started Python interpreter, and don’t just continue from the session described above: import logging logging.basicConfig(filename='example.log', encoding='utf-8', level=logging.DEBUG) logging.debug('This message should go to the log file') logging.info('So should this') logging.warning('And this, too') logging.error('And non-ASCII stuff, too, like Øresund and Malmö') Changed in version 3.9: The encoding argument was added. In earlier Python versions, or if not specified, the encoding used is the default value used by And now if we open the file and look at what we have, we should find the log messages: DEBUG:root:This message should go to the log file INFO:root:So should this WARNING:root:And this, too ERROR:root:And non-ASCII stuff, too, like Øresund and Malmö This example also shows how you can set the logging level which acts as the threshold for tracking. In this case, because we set the threshold
to If you want to set the logging level from a command-line option such as: and you have the value of the parameter passed for getattr(logging, loglevel.upper()) to get the value which you’ll pass to # assuming loglevel is bound to the string value obtained from the # command line argument. Convert to upper case to allow the user to # specify --log=DEBUG or --log=debug numeric_level = getattr(logging, loglevel.upper(), None) if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s' % loglevel) logging.basicConfig(level=numeric_level, ...) The call to If you run the above script several times, the messages from successive runs are appended to the file example.log. If you want each run to start afresh, not remembering the messages from earlier runs, you can specify the filemode argument, by changing the call in the above example to: logging.basicConfig(filename='example.log', filemode='w', level=logging.DEBUG) The output will be the same as before, but the log file is no longer appended to, so the messages from earlier runs are lost. Logging from multiple modules¶If your program consists of multiple modules, here’s an example of how you could organize logging in it: # myapp.py import logging import mylib def main(): logging.basicConfig(filename='myapp.log', level=logging.INFO) logging.info('Started') mylib.do_something() logging.info('Finished') if __name__ == '__main__': main() # mylib.py import logging def do_something(): logging.info('Doing something') If you run myapp.py, you should see this in myapp.log: INFO:root:Started INFO:root:Doing something INFO:root:Finished which is hopefully what you were expecting to see. You can generalize this to multiple modules, using the pattern in mylib.py. Note that for this simple usage pattern, you won’t know, by looking in the log file, where in your application your messages came from, apart from looking at the event description. If you want to track the location of your messages, you’ll need to refer to the documentation beyond the tutorial level – see Advanced Logging Tutorial. Logging variable data¶To log variable data, use a format string for the event description message and append the variable data as arguments. For example: import logging logging.warning('%s before you %s', 'Look', 'leap!') will display: WARNING:root:Look before you leap! As you can see, merging of variable data into the event description message uses the old, %-style of string formatting. This is for backwards compatibility: the logging package pre-dates newer formatting options such as Changing the format of displayed messages¶To change the format which is used to display messages, you need to specify the format you want to use: import logging logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG) logging.debug('This message should appear on the console') logging.info('So should this') logging.warning('And this, too') which would print: DEBUG:This message should appear on the console INFO:So should this WARNING:And this, too Notice that the ‘root’ which appeared in earlier examples has disappeared. For a full set of things that can appear in format strings, you can refer to the documentation for LogRecord attributes, but for simple usage, you just need the levelname (severity), message (event description, including variable data) and perhaps to display when the event occurred. This is described in the next section. Displaying the date/time in messages¶To display the date and time of an event, you would place ‘%(asctime)s’ in your format string: import logging logging.basicConfig(format='%(asctime)s %(message)s') logging.warning('is when this event was logged.') which should print something like this: 2010-12-12 11:41:42,612 is when this event was logged. The default format for date/time display (shown above) is like ISO8601 or
RFC 3339. If you need more control over the formatting of the date/time, provide a datefmt argument to import logging logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p') logging.warning('is when this event was logged.') which would display something like this: 12/12/2010 11:46:36 AM is when this event was logged. The format of the datefmt argument is the same as supported by
Next Steps¶That concludes the basic tutorial. It should be enough to get you up and running with logging. There’s a lot more that the logging package offers, but to get the best out of it, you’ll need to invest a little more of your time in reading the following sections. If you’re ready for that, grab some of your favourite beverage and carry on. If your logging needs are simple, then use the above examples to incorporate logging into your own scripts, and if you run into problems or don’t understand something, please post a question on the comp.lang.python Usenet group (available at https://groups.google.com/forum/#!forum/comp.lang.python) and you should receive help before too long. Still here? You can carry on reading the next few sections, which provide a slightly more advanced/in-depth tutorial than the basic one above. After that, you can take a look at the Logging Cookbook. Advanced Logging Tutorial¶The logging library takes a modular approach and offers several categories of components: loggers, handlers, filters, and formatters.
Log event information is passed between loggers, handlers, filters and formatters in a
Logging is performed by calling methods on instances of the A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows: logger = logging.getLogger(__name__) This means that logger names track the package/module hierarchy, and it’s intuitively obvious where events are logged just from the logger name. The root of the hierarchy of loggers is called the root logger. That’s the logger used by the functions It is, of course, possible to log messages to different destinations. Support is included in the package for writing log messages to files, HTTP GET/POST locations, email via SMTP, generic sockets, queues, or OS-specific logging mechanisms such as syslog or the Windows NT event log. Destinations are served by handler classes. You can create your own log destination class if you have special requirements not met by any of the built-in handler classes. By default, no destination is set for any logging messages. You can specify a destination (such as console or file) by using The default format set by severity:logger name:message You can change this by passing a format string to Logging Flow¶The flow of log event information in loggers and handlers is illustrated in the following diagram. Loggers¶
The most widely used methods on logger objects fall into two categories: configuration and message sending. These are the most common configuration methods:
You don’t need to always call these methods on every logger you create. See the last two paragraphs in this section. With the logger object configured, the following methods create log messages:
Loggers have a concept of effective level.
If a level is not explicitly set on a logger, the level of its parent is used instead as its effective level. If the parent has no explicit level set, its parent is examined, and so on - all ancestors are searched until an explicitly set level is found. The root logger always has an explicit level set ( Child
loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed. (You can, however, turn off propagation by setting the propagate attribute of a logger to Handlers¶
The standard library includes quite a few handler types (see Useful Handlers); the tutorials use mainly There are very few methods in a handler for application developers to concern themselves with. The only handler methods that seem relevant for application developers who are using the built-in handler objects (that is, not creating custom handlers) are the following configuration methods:
Application code should not directly instantiate and use instances of Formatters¶Formatter objects configure the
final order, structure, and contents of the log message. Unlike the base logging.Formatter. __init__ (fmt=None, datefmt=None, style='%')¶If there is no message format string, the default is to use the raw message. If there is no date format string, the default date format is: with the milliseconds tacked on at the end. The If the Changed in version 3.2:
Added the The following message format string will log the time in a human-readable format, the severity of the message, and the contents of the message, in that order: '%(asctime)s - %(levelname)s - %(message)s' Formatters use a user-configurable function to convert the creation time of a record to a tuple. By default, Configuring Logging¶Programmers can configure logging in three ways:
For the reference documentation on the last two options, see Configuration functions. The following example configures a very simple logger, a console handler, and a simple formatter using Python code: import logging # create logger logger = logging.getLogger('simple_example') logger.setLevel(logging.DEBUG) # create console handler and set level to debug ch = logging.StreamHandler() ch.setLevel(logging.DEBUG) # create formatter formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') # add formatter to ch ch.setFormatter(formatter) # add ch to logger logger.addHandler(ch) # 'application' code logger.debug('debug message') logger.info('info message') logger.warning('warn message') logger.error('error message') logger.critical('critical message') Running this module from the command line produces the following output: $ python simple_logging_module.py 2005-03-19 15:10:26,618 - simple_example - DEBUG - debug message 2005-03-19 15:10:26,620 - simple_example - INFO - info message 2005-03-19 15:10:26,695 - simple_example - WARNING - warn message 2005-03-19 15:10:26,697 - simple_example - ERROR - error message 2005-03-19 15:10:26,773 - simple_example - CRITICAL - critical message The following Python module creates a logger, handler, and formatter nearly identical to those in the example listed above, with the only difference being the names of the objects: import logging import logging.config logging.config.fileConfig('logging.conf') # create logger logger = logging.getLogger('simpleExample') # 'application' code logger.debug('debug message') logger.info('info message') logger.warning('warn message') logger.error('error message') logger.critical('critical message') Here is the logging.conf file: [loggers] keys=root,simpleExample [handlers] keys=consoleHandler [formatters] keys=simpleFormatter [logger_root] level=DEBUG handlers=consoleHandler [logger_simpleExample] level=DEBUG handlers=consoleHandler qualname=simpleExample propagate=0 [handler_consoleHandler] class=StreamHandler level=DEBUG formatter=simpleFormatter args=(sys.stdout,) [formatter_simpleFormatter] format=%(asctime)s - %(name)s - %(levelname)s - %(message)s The output is nearly identical to that of the non-config-file-based example: $ python simple_logging_config.py 2005-03-19 15:38:55,977 - simpleExample - DEBUG - debug message 2005-03-19 15:38:55,979 - simpleExample - INFO - info message 2005-03-19 15:38:56,054 - simpleExample - WARNING - warn message 2005-03-19 15:38:56,055 - simpleExample - ERROR - error message 2005-03-19 15:38:56,130 - simpleExample - CRITICAL - critical message You can see that the config file approach has a few advantages over the Python code approach, mainly separation of configuration and code and the ability of noncoders to easily modify the logging properties. Warning The
The dictionary passed to
Note that the class
names referenced in config files need to be either relative to the logging module, or absolute values which can be resolved using normal import mechanisms. Thus, you could use either In Python 3.2, a new means of configuring logging has been introduced, using dictionaries to hold configuration information. This provides a superset of the functionality of the config-file-based approach outlined above, and is the recommended configuration method for new applications and deployments. Because a Python dictionary is used to hold configuration information, and since you can populate that dictionary using different means, you have more options for configuration. For example, you can use a configuration file in JSON format, or, if you have access to YAML processing functionality, a file in YAML format, to populate the configuration dictionary. Or, of course, you can construct the dictionary in Python code, receive it in pickled form over a socket, or use whatever approach makes sense for your application. Here’s an example of the same configuration as above, in YAML format for the new dictionary-based approach: version: 1 formatters: simple: format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s' handlers: console: class: logging.StreamHandler level: DEBUG formatter: simple stream: ext://sys.stdout loggers: simpleExample: level: DEBUG handlers: [console] propagate: no root: level: DEBUG handlers: [console] For more information about logging using a dictionary, see Configuration functions. What happens if no configuration is provided¶If no logging configuration is provided, it is possible to have a situation where a logging event needs to be output, but no handlers can be found to output the event. The behaviour of the logging package in these circumstances is dependent on the Python version. For versions of Python prior to 3.2, the behaviour is as follows:
In Python 3.2 and later, the behaviour is as follows:
To obtain the pre-3.2 behaviour, Configuring Logging for a Library¶When developing a library which uses logging, you should take care to document how the library uses logging - for example, the names of loggers used. Some consideration also needs to be given to its logging configuration. If the using application does not use logging, and library code makes logging calls, then (as described
in the previous section) events of severity If for some reason you don’t want these messages printed in the absence of any logging configuration, you can attach a do-nothing handler to the top-level logger for your library. This avoids the message being printed, since a handler will always be found for the library’s events: it just doesn’t produce any output. If the library user configures logging for application use, presumably that configuration will add some handlers, and if levels are suitably configured then logging calls made in library code will send output to those handlers, as normal. A do-nothing handler is included in the logging package: import logging logging.getLogger('foo').addHandler(logging.NullHandler()) should have the desired effect. If an organisation produces a number of libraries, then the logger name specified can be ‘orgname.foo’ rather than just ‘foo’. Note It is strongly advised that you do not add any handlers other than Logging Levels¶The numeric values of logging levels are given in the following table. These are primarily of interest if you want to define your own levels, and need them to have specific values relative to the predefined levels. If you define a level with the same numeric value, it overwrites the predefined value; the predefined name is lost.
Levels can also be associated with loggers, being set either by the developer or through loading a saved logging configuration. When a logging method is called on a logger, the logger compares its own level with the level associated with the method call. If the logger’s level is higher than the method call’s, no logging message is actually generated. This is the basic mechanism controlling the verbosity of logging output. Logging messages are encoded as
instances of the Logging messages are subjected to a dispatch mechanism through the use of handlers, which are instances
of subclasses of the Just as for loggers, handlers can have levels associated with them. A
handler’s level acts as a filter in the same way as a logger’s level does. If a handler decides to actually dispatch an event, the Custom Levels¶Defining your own levels is possible, but should not be necessary, as the existing levels have been chosen on the basis of practical experience. However, if you are convinced that you need custom levels, great care should be exercised when doing this, and it is possibly a very bad idea to define custom levels if you are developing a library. That’s because if multiple library authors all define their own custom levels, there is a chance that the logging output from such multiple libraries used together will be difficult for the using developer to control and/or interpret, because a given numeric value might mean different things for different libraries. Useful Handlers¶In addition to the base
The
Logged messages are formatted
for presentation through instances of the For formatting multiple messages in a batch, instances of When filtering based on logger level and/or handler level is not enough, instances of The basic
Exceptions raised during logging¶The logging package is designed to swallow exceptions which occur while logging in production. This is so that errors which occur while handling logging events - such as logging misconfiguration, network or other similar errors - do not cause the application using logging to terminate prematurely.
The default implementation of Note The default value of Using arbitrary objects as messages¶In the preceding sections and examples, it has been assumed that the message passed
when logging the event is a string. However, this is not the only possibility. You can pass an arbitrary object as a message, and its Optimization¶Formatting of message arguments is deferred until it cannot be avoided. However,
computing the arguments passed to the logging method can also be expensive, and you may want to avoid doing it if the logger will just throw away your event. To decide what to do, you can call the if logger.isEnabledFor(logging.DEBUG): logger.debug('Message with %s, %s', expensive_func1(), expensive_func2()) so that if the logger’s threshold is set above Note In some cases, There are other optimizations which can be made for specific applications which need more precise control over what logging information is collected. Here’s a list of things you can do to avoid processing during logging which you don’t need:
Also note that the core logging module only includes the basic handlers. If you don’t import How do you create a log in Python?Here's an example:. import logging logging. basicConfig(level=logging. ... . DEBUG:root:This will get logged.. import logging logging. basicConfig(filename='app.log', filemode='w', format='%(name)s - %(levelname)s - %(message)s') logging. ... . root - ERROR - This will get logged to a file.. ERROR:root:This is an error message.. How do you define a log value?logarithm, the exponent or power to which a base must be raised to yield a given number. Expressed mathematically, x is the logarithm of n to the base b if bx = n, in which case one writes x = logb n. For example, 23 = 8; therefore, 3 is the logarithm of 8 to base 2, or 3 = log2 8.
How do you do log 2 in Python?Use math. log2() to compute the log base 2 of a number
Call math. log2(x) to return the base 2 logarithm of a number x as a float.
How do you define a natural log in Python?Natural logarithm log is the inverse of the exp(), so that log(exp(x)) = x. The natural logarithm is log in base e.
|