The apply function in Pandas is used to run a custom operation on data and create a transformed output. It is commonly used when built-in Pandas operations are not enough and you need logic that is specific to your dataset. Apply is useful for cleaning values, creating new columns, categorising records, and performing row-wise or column-wise transformations.
There are two common ways apply is used:
- Applying to a Series (a single column)
This is the most common use case. You apply a function to each value in a column. For example, you can convert text to a standard format, extract part of a string, or create a label based on rules. This is often used to build new columns such as “High” or “Low” based on a numeric threshold, or to standardise categories like city names. - Applying to a DataFrame (rows or columns)
You can apply a function across rows or across columns. Row-wise apply is used when a calculation depends on multiple columns together, such as creating a new field based on both sales and cost, or generating a combined identifier from multiple fields. Column-wise apply is less common in analysis but can be used for transformations across many columns.
Important points to remember:
- Apply can be slower on very large datasets compared to vectorised operations, so use it when necessary and keep the function simple.
- When your logic is simple, methods like map, replace, where, or vectorised string operations are often faster and cleaner.
- It is a good practice to test your function on a small sample before applying it to the full dataset.
Apply is valuable because it gives you flexibility to implement real business rules directly inside your data preparation and analysis workflow.
