Poisson Distribution

The Poisson distribution is a discrete probability distribution that represents the number of events occurring in a fixed interval of time or space. The Poisson distribution is particularly suitable for modelling rare events or situations where events occur independently and at a constant average rate.

The Poisson distribution is used through the world but it’s focused in various fields such as Probability Theory, Epidemiology, Insurance, Telecommunications, Queueing Theory, and Reliability Engineering. Additionally, it finds applications in Machine Learning and Data Science for tasks such as anomaly detection and count-based analysis.

What is the Poisson Distribution

The Poisson distribution is a discrete probability distribution that describes the number of events that occur within a fixed interval of time or space. It is named after the French mathematician Siméon Denis Poisson.

The probability mass function (PMF) of the Poisson distribution is given by:

Where:

• P(X=k) is the probability of observing k events.
• λ is the average rate of event occurrences in the given interval.
• e is the base of the natural logarithm (approximately equal to 2.71828).
• k is a non-negative integer, representing the number of events.

The Poisson Distribution Parameters

The Poisson distribution requires only two parameters for it to be successfully applied. The first argument is typically referred to as k. k is the specific number of events you wish to find the probability of. The other parameter is referred to as λ, it is the average rate of events in an interval. K and λ need to be non-negative integers, this means that its k and λ can be zero as well as any positive integer.

If λ is 0, it implies that no events are expected on average in the given interval. If λ is positive, it represents a positive average event rate. This also shows why λ has to be non-negative, as you cannot have a negative number of events occurring.

You can use the Poisson distribution to calculate the probability of observing these events within the given interval. It’s essential to consider scenarios where the number of events can be 0 because the Poisson distribution is used to model rare and random events, and sometimes, no events can occur within a specific interval.

Key Characteristics of the Poisson Distribution

The Poisson distribution is a probability distribution commonly used to model rare and independent events. The Poisson distribution has a memoryless property and is suitable for situations where events are infrequent. When summing independent Poisson variables, the result is also a Poisson-distributed variable.

The Poisson Distribution can approximate the binomial distribution under certain conditions. Notably, both the mean and variance of a Poisson distribution equal λ, and it’s positively skewed.

The Poisson Distribution in MatDeck

• The Poisson distribution can be used in a classical Python IDE via both MD Python and typical Python libraries.
• MatDeck Distribution functions can be used via the MatDeck IDE which allows for more conventional line by line coding.
• For those more comfortable with Python, Python functions can be directly called into MatDeck’s live document and be used like native MD functions.
• Both MD and Python functions can then be combined with 2D and 3D Graphs MD document without any code.

As we can see, you will need 3 additional libraries to model the Poisson distribution: numpy, matplotlib and seaborn. Once these libraries have been imported, you can use the radom.poisson() function to create the Poisson model, the model is then directly fed into the graph.

To model a Poisson distribution in MatDeck, you can use the poissondens() function which will return a single value or a vector depending on the data fed. The best way to plot a Poisson distribution is via the curve2d() function which will feed a known amount of x values from a range into the poissondens() function. The Poisson Model can then be directly added to the graph as well as dynamically changed in real time thanks to MatDeck’s live document.

Embedding Poisson Distribution in a MD Document

MatDeck allows users to natively utilize Python functions inside the MD Document. To do this, users create functions in a Python Script and save the script in the same directory as the MD Document. The user can then use the Python functions just like any MD Functions, allowing you to connect it to MD Toolboxes and features such as the 2D and 3D Graph. Below is an example of how you can use a unique Poisson distribution Python function in a MD Document. The following code was used for the Python Functions.

import math

def Poisson_Dist_Prob(k, diff):

    #Declaring variables to hold Poisson distribution probabilities

    less_than_k_minus_diff = 0

    more_than_k_plus_diff = 0

    # Calculate the sum of Poisson distribution probabilities less than 'k - diff

    for x in range(int(k - diff)):

        less_than_k_minus_diff += (k**x * math.exp(-k)) / math.factorial(x)

    # Calculate the sum of Poisson distribution probabilities more than 'k - diff'

    for x in range(int(k + diff + 1), int(k + 2 * diff)):

        more_than_k_plus_diff += (k**x * math.exp(-k)) / math.factorial(x)

    return less_than_k_minus_diff + more_than_k_plus_diff

This code represented the Poisson_Dist_Prob() function, which we can see being used below in an MD Document.

The Poisson Distribution Function

The poissondist function is used to express the probability of a given number of events occurring in a fixed interval of time and/or space if these events occur with a known average rate and independently of the time since the last event. It takes two arguments: Integer parameter k and Real parameter lambda (variance of x). Here we can see what the poissondist function looks like when plotted in a MD Document.

The Poisson Distribution for Artificial Intelligence

The Poisson distribution is a fundamental statistical concept with numerous practical applications in artificial intelligence The Poisson distribution is particularly valuable in scenarios involving rare events occurring over fixed intervals. Specifically, the Poisson distribution is used for tasks such as anomaly detection, as it helps identify unusual events like cybersecurity threats or network intrusions by modelling expected event rates.

Similarly, the Poisson distribution is applied in Natural Language Processing for analysing word frequencies in text, which assists in tasks like sentiment analysis and topic modelling.

In recommendation systems, the Poisson distribution is used to model user interactions with items, enabling personalized recommendations.

In the context of event prediction, the Poisson distribution is utilized to forecast future occurrences in time series data, relevant to domains like website traffic analysis and demand prediction.

Additionally, the Poisson distribution is a valuable tool in optimizing resource allocation, queuing systems, and sensor data analysis. For instance, The Poisson distribution can help in the efficient allocation of resources in server farms or the analysis of sensor data in IoT applications. In healthcare, the Poisson distribution is used to model disease outbreak frequencies and patient admission rates, facilitating epidemiological analysis and resource allocation.

The Poisson Distribution for Machine Learning

The Poisson distribution plays a pivotal role in machine learning across several domains. It excels in count data modelling, making it suitable for tasks like tracking email volumes, customer interactions, and manufacturing defects. Additionally, in time series forecasting, particularly for predicting future event counts in areas such as website traffic and stock prices, the Poisson distribution shines.

The Poisson distribution is heavy used due to its simplicity, scalability, and robustness, making it accessible to learners and adaptable to datasets of all sizes. Furthermore, it’s versatile, capable of handling over-dispersion through related distributions like the Negative Binomial.

The Poisson Distribution in Poisson Flow Generative Models

The Poisson distribution is a fundamental component in Poisson Flow Generative Models (PFGMs) as it plays a pivotal role in modelling the electric field that transforms data distributions into uniform angular distributions. In PFGMs, the charge distribution of data points generates an electric field that follows Poissonian dynamics. As data distributions evolve through time according to the Poisson field, they transition from their original forms into uniform angular distributions. This transformation, facilitated by the Poisson dynamics, enables the generation of novel data samples by following the electric field lines in reverse time.

The Poisson distribution’s probabilistic principles are harnessed to create an invertible mapping between a simple distribution, typically Gaussian, and the data distribution, making PFGMs a powerful tool in generative AI, where understanding and manipulating probabilistic processes are at the core of generating diverse and high-quality data.

Where is the Poisson Distribution Used

The Poisson Distribution is used in various fields and applications where random events or occurrences can be described and analysed. Some of the common areas where the Poisson distribution is applied include:

• Queueing Theory
• Insurance
• Epidemiology
• Environmental Science
• Quality Control
• Public Health
• Finance

 The Poisson Distribution in Actuarial Science

In actuarial science, the Poisson distribution is extensively used for modelling and analysing random events related to risk and insurance:

• Claims Modelling: Predict the number of insurance claims to estimate reserves and policy pricing.
• Risk Assessment: Evaluate rare events like accidents and disasters for portfolio risk analysis.
• Mortality Analysis: Model death counts for life insurance calculations.
• Loss Reserving: Estimate future losses and financial reserves for insurers.
• Policyholder Behaviour: Analyse lapses and surrenders to project policy cash flows.
• Aggregate Loss Modelling: Assess total losses considering event frequency and severity.
• Reinsurance Analysis: Evaluate reinsurance needs based on expected claims and risk exposure.

The Poisson Distribution in Insurance

In insurance, the Poisson distribution is crucial for modelling risk-related events, like insurance claims, risk assessment for rare events, mortality analysis, loss reserving, and reinsurance analysis. The Poisson distribution plays a pivotal role in estimating reserves, pricing policies, assessing portfolio risk, and ensuring financial solvency.

The Poisson Distribution in Queuing Theory

The Poisson distribution is a key tool for modelling various stochastic processes, like customer arrivals at service centres, call arrivals at call centres, and store checkout patterns. It’s vital for optimizing resources and improving service efficiency by accurately modelling arrivals and service times, informing staffing and system performance decisions.

Poisson Distribution Assumptions

The key assumptions of the Poisson distribution are:

• Independence: The events being counted must be independent, meaning the occurrence of one event should not affect the occurrence of another.
• Fixed Average Rate: The Poisson distribution assumes that events occur at a constant average rate over a fixed interval of time or space.
• Rare Events: The Poisson distribution is appropriate when the probability of more than one event occurring in an infinitesimally small interval is negligible. In other words, it is used for rare events.
• Countable Events: The events being counted should be discrete and easily countable, such as the number of emails received in an hour or the number of accidents at a specific intersection in a day.

Poisson Distribution Median

The median of a Poisson distribution is a value that separates the distribution into two equal halves, meaning that half of the probability mass is less than or equal to the median, and the other half is greater than or equal to the median. For a Poisson distribution with parameter λ the median is typically estimated as:

This formula is an approximation, and it becomes increasingly accurate as λ (the mean of the Poisson Dist) increases. For small values of λ, it may not provide an exact median value, but it’s a reasonable estimate. It’s a common and convenient way to estimate the median of a Poisson Dist.

Keep in mind that for integer values of λ, the median will be an integer. If you want a more accurate median estimate, you may use numerical methods or software to calculate it directly.

Expected Value of Poisson Distribution

The expected value, often referred to as the mean, of a Poisson distribution is a fundamental statistical measure that represents the average number of events that one can expect to occur within a fixed interval of time or space. It is denoted as E(X) or μ and is calculated as the product of the rate parameter λ, which characterizes the average rate at which events occur, and the interval length. Mathematically, E(X) = λ. The expected value serves as a critical point of reference for understanding the central tendency of a Poisson-distributed random variable.

Poisson Distribution Variance

The variance of a Poisson distribution is directly related to its rate parameter (λ). The variance (Var) of a Poisson distribution is equal to its rate parameter λ:

Var(X) = λ

In other words, the variance of a Poisson distribution is equal to its mean or expected value.

Poisson Distribution

In MatDeck, the poissondist function is used calculate the cumulative distributions function for the Poisson distribution in MatDeck. Its is incredibly accessible to all levels of programmers and only requires 2 arguments.

The first argument is the data for which you would like to predict the cumulative distribution function (CDF) for the Poisson distribution, it can be a single number or it could be a list of numbers. The second argument is the variance of the data, this can be calculated in MatDeck with the variance function and plugged straight into the poissondist function. MatDeck will then automatically calculate whether the CDF of your input data. We can the poissondist function in use below, within the curve2d function.

Here is a more comprehensive look at all the Poisson probability distribution functions:

• poisson-dens function

Inversing the Poisson Distribution

The poissoninv function is used to calculate he inverse cumulative distribution function (CDF) for the Poisson distribution, just like the poissondist function the poissoninv function requires the exact same arguments.

The first argument is the input data which we will apply the inverse cumulative distribution function (CDF) for the Poisson distribution to. The second argument is the variance. Once again, this can be calculated by the variance function and be directly used in the poissoninv function.

Uses of the Poisson Distribution

The Poisson distribution is used in all fields in which a discrete variable needs to be predicted within in a given timeframe or measure of space, because of this the Poisson distribution is preferred over other distributions in certain scenarios because:

• It is suitable for modelling count data.
• It is applicable when events occur infrequently.
• It assumes memorylessness, making it suitable for independent events.
• It has a simple parameterization and is easy to work with mathematically.
• It is analytically tractable for calculating probabilities and statistics.

However, it may not be appropriate if events are not independent or occur at varying rates. The choice of distribution depends on the specific data characteristics and research question.

Poisson Distributions in Python

All MD Statistics and Mathematical functions are available to be used in Python via MD Python. The MD Python bindings allow python users to natively call and use MatDeck functions in their work, this allows them to access C++ speed with the simple syntax of Python. It gives them unparalleled access to the MatDeck library, without them needing any experience to excel.