# Entry

You wrote a Dockerfile, built an image, and everything works. But then you notice that the image is over a gigabyte in size, rebuilding takes several minutes for even the smallest changes, and each press or pull feels painfully sluggish.

This is not unusual. These are the default results if you write Dockerfiles without thinking about base image selection, build context, and caching. You don’t need a complete overhaul to fix this. A few targeted changes can shrink your image by 60-80% and turn most rebuilds from minutes to seconds.

In this article, we’ll cover five practical techniques that will assist you learn how to make your Docker images smaller, faster, and more effective.

# Prerequisites

To follow along you will need:

• Docker installed
• Basic knowledge Dockerfiles and docker build order
• Python project with requirements.txt file (the examples employ Python, but the rules apply to any language)

# Select Slim or Alpine base images

Each Dockerfile starts with a FROM statement that selects a base image. This base image is the foundation on which your application is built, and its size becomes the minimum image size before adding one line of your own code.

For example, an official python:3.11 image is a full Debian-based image that includes compilers, tools, and packages that most applications never employ.

# Full image — everything included
FROM python:3.11

# Slim image — minimal Debian base
FROM python:3.11-slim

# Alpine image — even smaller, musl-based Linux
FROM python:3.11-alpine

Now build an image from each of them and check the sizes:

docker images | grep python

If you change one line in your Dockerfile, you’ll see a difference of several hundred megabytes. So which one should you employ?

• slim is the safer default for most Python projects. Removes unnecessary tools but retains C libraries that many Python packages need to install properly.
• alpine is even smaller, but uses a different C library — muscle instead glibc – which may cause compatibility issues with some Python packages. So you can spend more time debugging failed pip installs than you save on image size.

Rule of thumb: start with python: 3.1x-slim. Switch to Alpine only if you are sure your dependencies are compatible and need additional size reduction.

// Tier ordering to maximize cache

Docker builds images layer by layer, one statement at a time. Once the layer is built, Docker caches it. If nothing changes in the next build that would affect the layer, Docker will reuse the cached version and skip rebuilding it.

Hook: if a layer changes, each subsequent layer will be invalidated and built anew.

This is very crucial when installing dependencies. Here’s a common mistake:

# Bad layer order — dependencies reinstall on every code change
FROM python:3.11-slim

WORKDIR /app

COPY . .                          # copies everything, including your code
RUN pip install -r requirements.txt   # runs AFTER the copy, so it reruns whenever any file changes

Every time you change a single line in the script, Docker invalidates COPY . . layer and then reinstalls all dependencies from scratch. In design with bulky requirements.txtthese are minutes lost on reconstruction.

The solution is uncomplicated: first copy what changes at least.

# Good layer order — dependencies cached unless requirements.txt changes
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .           # copy only requirements first
RUN pip install --no-cache-dir -r requirements.txt   # install deps — this layer is cached

COPY . .                          # copy your code last — only this layer reruns on code changes

CMD ["python", "app.py"]

Now that you’ve changed app.pyDocker reuses the cached pip layer and only re-runs the final version COPY . ..

Rule of thumb: order yours COPY AND RUN instructions from least frequently changed to most frequently changed. Dependencies before code, always.

# Using multi-stage builds

Some tools are only needed at the build stage – compilers, test runners, build dependencies – but they end up in the final image anyway, populating it with elements that the running application never touches.

Multi-stage builds solve this problem. You employ one step to build or install everything you need, then copy only the finished result into a tidy, minimal final image. Build tools never make it into the image you upload.

Here is a Python example where we want to install dependencies but keep the final image:

# Single-stage — build tools end up in the final image
FROM python:3.11-slim

WORKDIR /app

RUN apt-get update && apt-get install -y gcc build-essential
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
CMD ["python", "app.py"]

Now with multi-step build:

# Multi-stage — build tools stay in the builder stage only

# Stage 1: builder — install dependencies
FROM python:3.11-slim AS builder

WORKDIR /app

RUN apt-get update && apt-get install -y gcc build-essential

COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# Stage 2: runtime — tidy image with only what's needed
FROM python:3.11-slim

WORKDIR /app

# Copy only the installed packages from the builder stage
COPY --from=builder /install /usr/local

COPY . .

CMD ["python", "app.py"]

The gcc and compilation tools – needed to compile some Python packages – disappeared from the final image. The application still runs because the compiled packages were copied. The build tools themselves remained in the builder stage, which Docker discards. This pattern is even more crucial in Go or Node.js projects, where compiler or node modules hundreds of megabytes in size can be completely excluded from the provided image.

# Cleaning in the installation layer

When installing system packages using apt-getthe package manager takes lists of packages and caches files you don’t need at runtime. If you delete them in a separate file RUN instructions still exist in the middleware, and Docker’s layering system means they still affect the final image size.

To actually remove them, the cleaning must occur in the same place RUN instructions as installation.

# Cleanup in a separate layer — cached files still bloat the image
FROM python:3.11-slim

RUN apt-get update && apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/* # already committed in the layer above

# Cleanup in the same layer — nothing is committed to the image
FROM python:3.11-slim

RUN apt-get update && apt-get install -y curl 
    && rm -rf /var/lib/apt/lists/*

The same logic applies to other package managers and momentary files.

Rule of thumb: everyone apt-get install should be followed && rm -rf /var/lib/apt/lists/* in the same RUN order. Make it a habit.

# Implementing .dockerignore files

When you run docker buildDocker sends everything in the build directory to the Docker daemon as the build context. This happens before any instructions in the Dockerfile are run, and often contains files that you almost certainly don’t want in your image.

Without .dockerignore file, you send the entire project folder: .git history, virtual environments, local data files, test devices, editor configurations, and more. This slows down each build and runs the risk of copying sensitive files into the image.

AND .dockerignore file works exactly like .gitignore; tells Docker which files and folders should be excluded from the build context.

Here is a sample, although truncated, .dockerignore for a typical Python data project:

# Python
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
*.egg-info/

# Virtual environments
.venv/
venv/
env/

# Data files (don't bake huge datasets into images)
data/
*.csv
*.parquet
*.xlsx

# Jupyter
.ipynb_checkpoints/
*.ipynb

...

# Tests
tests/
pytest_cache/
.coverage

...

# Secrets — never let these into an image
.env
*.pem
*.key

This significantly reduces the data sent to the Docker daemon before compilation begins. For huge data projects where parquet files or raw CSV files reside in the project folder, this may be the single biggest win of all five practices.

It is worth paying attention to the safety aspect. If your project folder contains .env files with API keys or database credentials, forgetting .dockerignore means that these secrets can be incorporated into your image – especially if you have extensive knowledge COPY . . instruction.

Rule of thumb: Always add .env and any authentication files for .dockerignore except for data files that do not need to be blended into the image. Utilize also Docker secrets for sensitive data.

# Abstract

None of these techniques require advanced Docker knowledge; they are more habits than techniques. Utilize them consistently and your images will be smaller, your builds will be faster, and your deployments will be cleaner.

Joyful coding!

Priya C’s girlfriend is a software developer and technical writer from India. He likes working at the intersection of mathematics, programming, data analytics and content creation. Her areas of interest and specialization include DevOps, data analytics and natural language processing. She enjoys reading, writing, coding and coffee! He is currently working on learning and sharing his knowledge with the developer community by writing tutorials, guides, reviews, and more. Bala also creates captivating resource overviews and coding tutorials.

“[Data center operators’] we believe that the value delivered by servers far outweighs the costs of constantly operating these inefficient power plants,” says Koomey.

Gas projects developed as part of Project Stargate, a massive multi-company artificial intelligence effort that was originally launched to build infrastructure for OpenAI, also represent a potential carbon bomb on WIRED’s list. Stargate campuses are being built in multiple states, including Texas, New Mexico, Ohio and Wisconsin. Permitting documents for just three natural gas projects linked to Stargate – one to power a data center campus near the project’s headquarters in Abilene, Texas, and two to power Project Jupiter at the New Mexico campus – show they have a combined potential to emit more than 24 million tons of greenhouse gases annually.

“We are committed to protecting payers while building the infrastructure needed for American AI leadership,” OpenAI spokesman Aaron McLear said in a statement. “Where natural gas is needed to provide reliable energy in the near future, we are working with partners to leverage modern, efficient generation while helping to accelerate clean energy and grid modernization.”

Oracle spokeswoman Julia Allyn Fishel told WIRED that a “modification” is currently underway to Project Jupiter “that is expected to significantly reduce greenhouse gas emissions.” The company did not provide new emissions estimates, which the New Mexico Department of the Environment has not yet made public.

“Oracle is committed to self-funding energy costs by implementing the best energy solutions for every community, so that our AI data centers do not impact ratepayer bills and electric grid reliability,” Fishel said in a statement.

The fourth gas plant at Stargate’s main campus in Abilene has the potential to produce more than 7.8 million tons of carbon dioxide equivalent each year, according to application documents. This power plant is being built by Crusoe for Microsoft’s employ. Companies announced in tardy March, Crusoe will be constructing fresh buildings on the Abilene campus, including a power plant, to support Microsoft’s artificial intelligence infrastructure. (Microsoft declined to comment.)

There are projects with an even larger potential carbon footprint than Stargate. Outside of Amarillo, Texas, White House darling Fermi is building what President Donald J. Trump calls the Advanced Energy and Intelligence Campus, a data center campus with a target capacity of 17 gigawatts. Fermi continues to emphasize the use of so-called “tidy” natural gas. However, the documents show that the maximum emissions of both gas projects combined could amount to over 40.3 million tons of CO₂ equivalents per year, more than the annual emissions of all energy sources in the state of Connecticut.

About five hours south of Amarillo, near the town of Fort Stockton, Pacifico Energy is expanding claims is the largest single energy project in the country: a 7.2-gigawatt data center campus powered by a gas project that could emit more than 33 million tons of greenhouse gases annually. (Pacifico did not respond to a request for comment.)

# Entry

Most data scientists are learning pandas by reading tutorials and copying working patterns.

This is a good solution to start with, but it often causes beginners to develop bad habits. Exploit iterrows() loops, intermediate variable assignments and repeatables merge() calls are some examples of code that is technically exact, but slower than necessary and harder to read than it should be.

The following patterns are not edge cases. They cover the most common everyday operations in data science, such as filtering, transforming, joining, grouping, and calculating conditional columns.

In each of them, there is a common approach and a better approach, and the distinction is usually one of awareness, not complexity.

These six have the greatest impact: combining methods, pipe() pattern, capable joins and merges, clustering optimizations, vectorized conditional logic, and performance pitfalls.

# Chain of methods

Indirect variables can make your code more organized, but they often just add noise. Chain of methods lets you write a sequence of transformations as a single expression that reads naturally and avoids naming objects that don’t require unique identifiers.

Instead:

df1 = df[df['status'] == 'dynamic']
df2 = df1.dropna(subset=['revenue'])
df3 = df2.assign(revenue_k=df2['revenue'] / 1000)
result = df3.sort_values('revenue_k', ascending=False)

You write this:

result = (
    df
    .query("status == 'active'")
    .dropna(subset=['revenue'])
    .assign(revenue_k=lambda x: x['revenue'] / 1000)
    .sort_values('revenue_k', ascending=False)
)

Lambda in assign() is significant here.

While linking, the current file status DataFrame cannot be accessed by name; you have to operate lambda to refer to it. The most common cause of broken chains is forgetting, which usually results in: NameError or an invalid reference to a variable that was defined earlier in the script.

Another mistake to be aware of is usage inplace=True inside the chain. Methods from inplace=True return Nonewhich immediately breaks the chain. When writing string code, in-place operations should be avoided because they provide no memory benefits and make the code hard to track.

# Pipe() pattern.

When one of your transformations is intricate enough to deserve its own function, using pipe() allows you to keep it inside the chain.

pipe() passes DataFrame as the first argument of any call:

def normalize_columns(df, cols):
    df[cols] = (df[cols] - df[cols].mean()) / df[cols].std()
    return df

result = (
    df
    .query("status == 'active'")
    .pipe(normalize_columns, cols=['revenue', 'sessions'])
    .sort_values('revenue', ascending=False)
)

This allows you to keep intricate transformation logic within a named, testable function while preserving the chain. Each pipelined function can be tested individually, which becomes a challenge when the logic is hidden in an extensive chain.

Practical value pipe() goes beyond appearance. Dividing the processing pipeline into labeled functions and combining them pipe() enables self-documentation of the code. Anyone who reads the sequence can understand each step from the function name without having to analyze the implementation.

It also makes it easier to swap or skip steps when debugging: if you comment out one pipe() call, the rest of the chain will continue to function smoothly.

# Productive combining and merging

One of the most overused features of pandas is to combine(). The two most common errors are many-to-many joins and noiseless row inflation.

If both dataframes have duplicate values ​​in the join key, merge() performs the Cartesian product of these rows. For example, if the join key is not unique on at least one side, joining the 500-row “users” table with the “events” table could result in millions of rows.

This does not cause an error; it just produces DataFrame which seems correct but is larger than expected until you check its shape.

The solution is validate parameter:

df.merge(other, on='user_id', validate="many_to_one")

It elevates MergeError immediately if the many-to-one assumption is violated. Exploit “one_to_one”, “one_to_many” or “many_to_one” depending on what you want from the linking.

The indicator=True parameter is equally useful for debugging:

result = df.merge(other, on='user_id', how='left', indicator=True)
result['_merge'].value_counts()

This parameter adds _merge a column showing whether each row comes from ‘only_left’, ‘only_right’, or ‘both’. This is the fastest way to catch rows that didn’t match when you expected them to match.

In cases where both data frames share a common index, append() is faster than merge() because it works directly on the index instead of searching a specific column.

# Group optimizations

When using A GroupByone of the rarely used methods is transform(). The difference between agg() AND transform() depends on what shape you want to recover.

The agg() method. returns one row per group. On the other hand, transform() restores the same shape as the original DataFramewith each row filled with the aggregated value of a given group. This makes it ideal for adding group-level statistics as modern columns without having to merge them later. It’s also faster than the manual approach to aggregating and merging because pandas doesn’t have to align two data frames after the fact:

df['avg_revenue_by_segment'] = df.groupby('segment')['revenue'].transform('mean')

This will directly add the average revenue for each segment to each row. Same result with agg() would require calculating the average and then reconnecting based on the segment key, using two steps instead of one.

For category grouping columns, always operate observed=True: :

df.groupby('segment', observed=True)['revenue'].sum()

Without this argument, pandas calculates results for each category defined in the dtype column, including combinations that do not appear in the actual data. For gigantic data frames with many categories, this results in empty groups and unnecessary computations.

# Vectorized conditional logic

Using apply() With lambda function per-row is the least capable way to calculate conditional values. It avoids the C-level operations that speed up pandas by running the Python function independently on each line.

For binary conditions NumPy‘S np.where() is a direct replacement for:

df['label'] = np.where(df['revenue'] > 1000, 'high', 'low')

For many conditions np.select() deals with them cleanly:

conditions = [
    df['revenue'] > 10000,
    df['revenue'] > 1000,
    df['revenue'] > 100,
]
choices = ['enterprise', 'mid-market', 'small']
df['segment'] = np.select(conditions, choices, default="micro")

The np.select() the function directly maps the if/elif/else structure at vector speed, evaluating the conditions in order and assigning the first matching option. This is typically 50 to 100 times faster than the equivalent apply() on DataFrame with a million lines.

In the case of numerical categorization, the conditional assignment is completely replaced by pd.cut() (containers of equal width) i pd.qcut() (quantile-based bins) that automatically return a categorical column without the need to operate NumPy. Pandas takes care of everything, including labeling and boundary value handling when you pass the number of bins or the edges of the bins.

# Performance pitfalls

Some common patterns leisurely down pandas code more than anything else.

For example, iterrows() repeats DataFrame rows as (index, Series) pairs. This is an intuitive but leisurely approach. For DataFrame at 100,000 rows this function call can be 100 times slower than its vector counterpart.

The lack of efficiency comes from building a complete system Series object for each line and executing the Python code on it one by one. As soon as you notice yourself writing for _, row in df.iterrows()stop and think if np.where(), np.select()or a grouping operation can replace it. In most cases, one of them can do it.

Using apply(axis=1) is faster than iterrows() but it has the same problem: python-level execution for each line. For any operation that can be represented using NumPy’s or pandas’ built-in functions, the built-in method is always faster.

Object type columns are also an simple to miss source of slowness. When pandas stores strings as an object type d, operations on these columns are performed in Python rather than C. For low-cardinality columns such as status codes, region names, or categories, converting them to a categorical type can significantly speed up grouping and value_counts().

df['status'] = df['status'].astype('category')

Finally, avoid string assignments. Using df[df['revenue'] > 0]['label'] = 'positive' he could change the initial DataFramedepending on whether pandas generated the copy behind the scenes. The behavior is undefined. To operate .loc instead next to the logical mask:

df.loc[df['revenue'] > 0, 'label'] = 'positive'

It is unambiguous and awakens no SettingWithCopyWarning.

# Application

These patterns distinguish code that works from code that works well: it is capable enough to operate on real data, readable enough to maintain, and organized in a way that makes it simple to test.

Chain of methods i pipe() address readability, while linking and grouping patterns address correctness and efficiency. Vectorization logic and trap section address speed.

Most of the pandas code we review has at least two or three of these problems. They build up silently – there’s a leisurely loop here, an unacknowledged connection there, or an object type column that no one noticed. None of them cause obvious failures, so they persist. A sharp way to start is to repair them one at a time.

Nate Rosidi is a data scientist and product strategist. He is also an adjunct professor of analytics and the founder of StrataScratch, a platform that helps data scientists prepare for job interviews using real interview questions from top companies. Nate writes about the latest career trends, gives interview advice, shares data science projects, and discusses all things SQL.

Astronomy in mid-April enthusiasts will be able to enjoy one of the classic celestial spectacles. The meteor shower known as the Lyrids will lithe up the sky, especially in the Northern Hemisphere, and anyone will be able to see it with the naked eye, weather permitting – if they know where to look.

Lyrids began to appear on April 14, but their peak activity occurred on the night of April 21 and the early morning of April 22. according to NASA. During these hours, the shower will show 15 to 20 meteors per hour under dim skies.

The shower gets its name because the meteors appear to emerge from the constellation Lyra. Locating the radiant is uncomplicated if you apply an astronomical mapping app: just find Vega, the fifth brightest star in the sky, behind only Sirius, Canopus, Alpha Centauri A and Arcturus. Once you locate it, look around; the bright traces of the Lyrids will appear projected from this point due to the effect of perspective. Please remember that the human eye takes 20 to 30 minutes to adjust to the dim.

During its peak, the Moon will be in the early crescent phase, so there will be very little interference from its lithe. In a dim sky, meteors should stand out easily. The shower is usually perceptible from 10 p.m. until dawn, although early morning offers the best conditions. It’s best to stay away from lithe pollution and, if possible, observe from a high altitude. A trip to the mountains works well.

Each meteor shower has a different origin. In April, the Earth passes through a cloud of fragments left by comet C/1861 G1 (Thatcher) in its orbit around the Sun. This comet, discovered in 1861, takes approximately 415 years to complete its journey. Grains of ice and rock that it released centuries ago enter the atmosphere at high speeds and produce flares we know as Lyrids.

After the Lyrids, the calendar still offers several spectacles for those who follow the night sky. The Eta Aquarids will arrive in May along with the remnants of Halley’s Comet. The Perseids will appear in August, the Orionids will return in October, and the year will end with the Leonids in November and the Geminids in December. The latter is considered the most intense and reliable shower on the calendar.

This story originally appeared on WIRED in Spanish and was translated from Spanish.

US Federal According to the letter obtained by WIRED, the government’s central energy information agency plans to implement a mandatory, nationwide survey of data centers, focusing on their energy consumption. This survey would be the first attempt of its kind to collect basic information about data centers.

The letter was sent to Senators Elizabeth Warren and Josh Hawley on April 9 by the head of the Energy Information Administration, Tristan Abbey, and responds to an earlier inquiry from senators about EIA’s plans to obtain more information about data centers. WIRED reported on Hawley and Warren’s letter last month.

“Americans deserve to know how much energy data centers consume and what impact it has on their utility bills,” Warren told WIRED in a statement. “EIA’s mandatory survey is an important first step toward holding data centers accountable, but people are hurting right now. I’m pushing EIA to collect and share this data as quickly as possible.”

EIA told WIRED it had no details to provide beyond those included in the letter to senators.

The explosion of data centers across the United States has sparked a wave of public concern and proposed legislation to limit the employ of their resources and impose a moratorium on their construction. However, surprisingly little official data on the industry has been collected.

Most details about data center energy employ – a particular concern for many voters in the face of rising utility bills – are considered proprietary business information and are not usually made public. In response to the Trump administration’s incentives to protect ratepayers, many data center developers are now starting to build their own power sources, known as behind-the-meter power. These facilities – many of which are gas-powered – are raising up-to-date concerns about air pollution and climate change. (On Tuesday, the NAACP filed a lawsuit against xAI alleging that it ran gas turbines behind the meter at a data center in Mississippi without a permit and polluted the surrounding community. xAI did not immediately respond to a request for comment.)

The EIA conducts mandatory investigations of suppliers of various types of energy generation, including oil and gas production, electricity generation and renewable energy sources, as well as their industrial customers. In overdue March, the day before the senators sent their letter, EIA announced that it would conduct a pilot survey in three areas of the country experiencing intense data center development: Texas, Washington state and the northern Virginia/DC metropolitan area.

In the April 9 letter, Abbey says the agency will announce a second tranche of pilot studies “spanning at least three additional states.” Both studies will be completed by the end of September. Abbey writes that these two pilot studies represent “a necessary step in the methodological development of a nationwide mandatory study.”

According to the letter, the information EIA collects from data centers under these pilot programs includes not only information on annual electricity consumption, but also information on off-meter energy generation. The surveys, Abbey writes, will also include questions about the classification of different types of data centers; cooling systems; characteristics of the object, e.g. surface; and IT specifications, including data center energy efficiency metrics.

The letter still leaves many unanswered questions about the pilot structure.

According to the letter, the pilot program will not ask each respondent about the full set of metrics, but rather will tailor the questions “to the specific location of each data center facility.” The current remote control also asks 196 enterprises identified in three regions to choose only one location for which they will report indicators. EIA did not respond to questions about how it determined which locations should receive which questions, or whether it had set any requirements for surveying respondents about how they selected which data center locations to provide information about.

The EIA also did not respond to WIRED’s questions about when the second series of pilot studies are scheduled to begin, which states will be included, or the possible timing of a mandatory national-level study.

# Entry

So you’re a student or someone just starting to learn the operational side of app development. You’ve already taken the first step by developing and testing your app locally. Now you want to deploy it in the cloud so you can access it from anywhere. The problem is that cloud hosting can seem complicated and costly when you’re just starting out.

In this article, we’ll look at some of the simplest free platforms that allow you to host a Python web app or application programming interface (API) application without paying upfront. While these services have confined processing power, they are usually more than enough for your first toy project, personal demo, or just experimenting with deploying, monitoring, and basic application management.

# 1. Share AI apps with huggable facespaces

Hugging facial space is one of my favorite options for hosting Python applications, especially if you work on AI-related projects. It is very beginner-friendly and makes implementation less intimidating. You can run Built application by simply transferring files by pressing a button Git commits or even using the Hugging Face Command Line Interface (CLI).

This is particularly useful for machine learning and enormous language model (LLM) projects, but also supports Streamlined and Docker-based applications. This gives you some flexibility depending on how straightforward and custom your application is.

The default free hardware in Hugging Face Spaces provides 2 CPU cores, 16 GB of RAM, and 50 GB of volatile disk space, which is more than enough for many demos, prototypes, class projects, and compact experiments.

Please note that Spaces on the basic free CPU tier will automatically go to sleep after approximately 48 hours of inactivity, but will restart when someone revisits the app.

# 2. Deploy data applications with Streamlit Community Cloud

Streamlined social cloud was one of the first platforms I used when learning to implement web applications in Python. Along Herokuthis made the whole process much easier to understand. This is a great starting point for beginners because you can go from a local project to a working application without having to do too much setup.

While many people still think of Streamlit as just a dashboard tool, it has become a versatile way to create data applications, internal tools, and lightweight, interactive web applications in Python. Implementation experience is one of its greatest advantages because your GitHub the repository acts as a source of truth, and pushes to the repository are automatically reflected in the application.

Streamlit claims that with the free tier, all Community Cloud users share the same pool of resources, with approximate limits of 0.078 to 2 CPU cores, 690 MB to 2.7 GB of memory, and up to 50 GB of storage. It’s worth knowing that apps that are idle for 12 hours go to sleep, but can be woken up again when someone visits the app.

# 3. Deploy backend APIs with Render

To give back is a more complete hosting platform that allows you to deploy all kinds of web applications, including Python, Node.js, Ruby on Rails, and Docker-based services. This is a forceful option if you want to host e.g Flask Or FastAPI backend without configuring servers yourself.

The implementation process is very straightforward. You connect to a GitHub repository – although Render also supports it GitLab AND Bitbucket — and the platform handles the build and deployment process for you. This makes it a very beginner-friendly way to get the Python API online.

Render offers a free web services layer that is useful for testing ideas, hobby projects, and compact demos. The essential thing to know is that free internet services stop working after 15 minutes of inactivity, and when someone visits again, it can take up to a minute for the service to wake up again.

# 4. Run Python applications using Modal

Modal is one of my favorite contemporary platforms for running Python applications, especially when the project is a little more advanced than a straightforward demo. I used it for Model Context Protocol (MCP) backends, AI agents, and more sophisticated applications where I wanted something quickly without having to manage the infrastructure myself. One of the coolest parts is that you define the infrastructure in Python, so the whole developer experience feels much more natural if you’re already working in the Python ecosystem.

It is especially useful for machine learning workloads, background tasks, and back-end services. You can run Python functions, scheduled tasks, and web endpoints, making it versatile enough for APIs, asynchronous processing, and model inference.

The free tier is quite generous to start with. Modal’s Starter plan includes $30 per month in free credit, plus a confined number of network endpoints and cron jobs, which is usually enough for compact experiments, personal projects, and early prototypes.

# 5. Host full Python applications on PythonAnywhere

Python, anywhere is one of the most famed Python hosting platforms. It feels a bit more venerable school compared to newer tools, but it still gets the job done. One of the reasons people keep coming back to it is that it is built specifically for Python, so you can write code, manage files, open consoles, and deploy web applications from your browser without having to set up your own server.

This is a good option for straightforward Flask i Django projects, especially if you want an all-in-one environment rather than combining multiple separate services. For beginners, this can make learning much easier.

The free account is really useful for learning and compact projects. Currently free accounts include:

• One web app with one workflow.
• Two consoles.
• 512 MiB disk space and 100 CPU seconds.
• Apps run on yourusername.pythonanywhere.com subdomain, and free accounts confined outgoing Internet access.

# Summary

Here’s a quick comparison to aid you choose the right platform depending on the type of Python application you want to deploy.

Abid Ali Awan (@1abidaliawan) is a certified data science professional who loves building machine learning models. Currently, he focuses on creating content and writing technical blogs about machine learning and data science technologies. Abid holds a Master’s degree in Technology Management and a Bachelor’s degree in Telecommunications Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

Neuroscientists know this There is a link between loneliness and cognitive decline in older people, although the exact scale of this link is still tough to understand. A modern longitudinal study provides evidence that some people who feel lonely are more likely to have memory problems, although this does not necessarily mean that their brains are aging faster.

The report, published in the journal Aging & Mental Health, found that older adults with higher levels of loneliness performed worse on immediate and delayed recall tests. Yet the rate of decline in their memory over the six years was virtually the same as that of people who were not single.

“This suggests that loneliness may play a more significant role in the initial state of memory than in its progressive decline,” he added. he said Luis Carlos Venegas-Sanabria from the School of Medicine and Health Sciences at Universidad del Rosario, who led the research. “The study highlights the importance of addressing loneliness as an important factor in older adults’ cognitive performance.”

A six-year study of thousands of lonely people

The team analyzed data from the Study of Health, Aging and Retirements in Europe (SHARE), one of the most tough longitudinal databases for studying aging. For six years, researchers followed 10,217 adults aged 65 to 94 from 12 European countries. They assessed their level of loneliness and performance on memory tests.

Results show that age was the most essential determinant of memory level and the rate of its deterioration. From the age of 75, the results began to decline faster. After the age of 85, the decline became more pronounced. Depression and chronic diseases such as diabetes also lowered the initial score. Loneliness, while affecting baseline, did not accelerate the slope of cognitive decline.

The study also found that physical activity was associated with better initial memory scores. People who engaged in moderate or vigorous physical activity at least once a month remembered more words on immediate and delayed recall tests. This effect did not change the rate of decline, but it did raise the baseline level, which functions as a kind of “cognitive buffer.”

Although the study did not examine the causes of the association between loneliness and cognitive function, previous research has proposed likely mechanisms. Loneliness is often associated with less social interaction, which is a factor in cognitive performance. It is also associated with an increased risk of depression, which directly affects memory tests. Additionally, single people tend to have more health problems, such as hypertension and diabetes, which also affect cognitive function.

According to United Nations forecasts, by 2050, every sixth person in the world will be over 65 years of age. Societies are entering a stage where senior age will no longer be an exception, but will become the norm. Dementia, like other neurodegenerative diseases that appear with age, will be a grave challenge for health care facilities.

Latest cosmology the model combines two of the most eccentric ideas in up-to-date physics to explain the nature of obscure matter, an unseen substance that makes up about 85 percent of all matter in the universe. To understand this, you need to look beyond the Large Bang that we all know and consider two concepts that rarely intersect: cyclic universes and primordial black holes.

A different kind of multiverse

There are different versions of the “multiverse”. The most popular model – the Marvel Cinematic Universe – assumes that there are as many universes as possible and that these versions of reality are parallel. Physics proposes something more sober and mathematically coherent: cosmic reflection.

In this model, the universe is not born from a singularity, but expands, contracts and expands again in an infinite cycle. Each “universe” is not parallel, but sequential – that is, one arises from the ashes of the previous one.

Is it possible for something to survive the end of its universe and survive into the next one? According to the publication published in Physical inspection dYes. Author Enrique Gaztanaga, research professor at the Institute of Space Sciences in Barcelona, ​​shows that any structure larger than about 90 meters can survive the final collapse of the universe and survive the rebound. These “relics” would not only survive, but could also give rise to the gigantic, unexplained structures observed in the early stages of the up-to-date universe. What’s more, they may be the key to understanding obscure matter.

For decades, the dominant explanation for obscure matter was that it was an unknown particle or particles. But after years of experimentation without direct discoveries, physicists began to explore alternatives. One of them suggests that obscure matter is not an exotic particle, but an bountiful population of diminutive black holes that we overlook.

The idea is attractive, but it has a stern problem. For these black holes to explain obscure matter, they would have to have existed from the earliest moments of the universe, long before the first stars collapsed. There are indications that these objects may exist, but there is no convincing physical mechanism to explain their origin.

A universe born with black holes

This is where Gaztanaga’s newly proposed model shines. If cosmic reflections allowed compact structures to survive the collapse of the previous universe, then the current universe would have already been born with pre-existing black holes. They would not have to arise from extreme fluctuations or finely tuned inflationary processes, but would simply exist from the first moment.

This assumption has the potential to solve two mysteries at once: the origin of black holes and the nature of obscure matter. If this model is correct, obscure matter would not be the secret of the early universe, but rather the legacy of a cosmos older than ours.

“Much work remains to be done,” said Gaztanaga, also a researcher at the Institute of Cosmology and Gravity at the University of Portsmouth, in an article for the magazine Conversation. “These ideas need to be tested with data ranging from the gravitational wave background to galaxy surveys and precise measurements of the cosmic microwave background.”

“But the possibility is great,” he added. “The universe may not have started once, but it may have bounced back. And the dark structures that shape today’s galaxies may be relics from a time before the Big Bang.”

This story originally appeared on WIRED in Spanish and was translated from Spanish.

Original version With this story appeared in Quanta Magazine.

Imagine a bizarre training exercise: a group of runners start running on a circular track, each maintaining a unique, constant pace. Will every runner finish “alone” or relatively far from others at least once, regardless of speed?

Mathematicians guess the answer is yes.

The “lone runner” problem may seem basic and inconsequential, but it appears in many forms in mathematics. This is the equivalent of questions in number theory, geometry, graph theory, and more – about when you can get clear visibility in a field of obstacles, where billiard balls can move across a table, or how to organize a network. “It has so many facets. It touches on so many different areas of mathematics,” he said Maciej Beck San Francisco State University.

With just two or three runners, the evidence for this hypothesis is rudimentary. Mathematicians proved this with four runners in the 1970s, and by 2007. up to seven. However, over the past two decades, no one has been able to go further.

Then last year Matthieu Rosenfeldmathematician from the Laboratory of Computer Science, Robotics and Microelectronics in Montpellier, resolved the conjecture eight runners. And within a few weeks, a second-year student from Oxford University named Tanupat (Paul) Trakulthongchai built on Rosenfeld’s ideas to prove it nine and 10 runners.

The sudden progress has sparked renewed interest in the problem. “It’s a real milestone,” said Beck, who was not involved in the work. Adding just one factor makes proving this hypothesis “exponentially more difficult,” he said. “To go from seven runners to now 10 runners is amazing.”

Starting jump

In the beginning, the lone runner problem had nothing to do with running.

Instead, mathematicians were interested in a seemingly unrelated problem: how to utilize fractions to approximate irrational numbers like pi, a task that has a huge number of applications. In the 1960s, a graduate of Jörg M. Wills he guessed that there is a hundred-year-old method for this is optimal – that there is no way to improve it.

In 1998, a group of mathematicians he rewrote this supposition in the language of running. To talk N runners start from the same place on a circular track 1 unit long and each runs at a different constant speed. Wills’ hypothesis is equivalent to saying that every runner will always be lonely at some point, regardless of the speed of the other runners. More specifically, each runner will at some point be at least 1/N from another runner.

When Wills saw the article about the lone runner, he emailed one of the authors: Luis Goddyn Simon Fraser University to congratulate him on “this wonderful and poetic name.” (Goddyn’s response: “Oh, you’re still alive.”)

Mathematicians have also shown that the lone runner problem is equivalent to yet another question. Imagine an endless sheet of graph paper. Place a miniature square in the center of each grid. Then start at one of the corners of the grid and draw a straight line. (The line can point in any direction other than perfectly vertical or horizontal.) How substantial can the smaller squares be before the line must hit one?

As versions of the lone runner problem spread throughout mathematics, interest in the question increased. Mathematicians have proven different cases of the conjecture using completely different techniques. Sometimes they relied on the tools of number theory; other times they turned to geometry or graph theory.

# Entry

These advanced validation issues are insidious. They pass basic quality checks because the individual values ​​look good, but the underlying logic is broken. Manually controlling these issues is challenging. You need automated scripts that understand context, business rules, and relationships between data points. This article discusses five advanced Python validation scripts that catch subtle issues that basic checks miss.

You can download the code on GitHub.

# 1. Checking for continuity and patterns in time series

// Pain point

// What the script does

// How it works

The script parses the timestamp columns to infer the expected frequency, identifying gaps in the expected continuous sequences. Verifies that event sequences follow logical ordering rules, applies domain-specific rate checks, and detects seasonality violations. It also generates detailed reports showing time anomalies along with assessing the impact on business activities.

# 2. Semantic validation using business rules

// Pain point

Individual fields pass type validation, but the combination makes no sense. Here are some examples: a purchase order from the future with a completed delivery date in the past. An account marked as “new customer” but with a transaction history of five years. These semantic violations break business logic.

// What the script does

Verifies data against complicated business rules and domain knowledge. It checks multi-field conditional logic, checks stages and time progression, ensures respect for mutually exclusive categories, and flags logically impossible combinations. The script uses a rules engine that can express advanced business constraints.

// How it works

The script accepts business rules defined in a declarative format, evaluates complicated conditional logic across multiple fields, and checks state changes and workflow progress. It also checks the temporal consistency of business events, applies industry-specific domain rules, and generates violation reports categorized by rule type and business impact.

Download the semantic validity checker script

# 3. Detecting data drift and schema evolution

// Pain point

Data structure sometimes changes over time without documentation. Novel columns appear, existing columns disappear, data types change slightly, value ranges expand or contract, categorical values ​​create recent categories. These changes break downstream systems, invalidate assumptions, and cause hushed failures. Before you know it, months of corrupt data have accumulated.

// What the script does

Monitors datasets for structural and statistical drift over time. It tracks schema changes such as recent and deleted columns, type changes, detects changes in the distribution of numeric and categorical data, and identifies recent values ​​in supposedly established categories. Flags changes in data ranges and restrictions and warns when statistical properties deviate from baseline values.

// How it works

The script creates baseline profiles of the structure and statistics of the dataset, periodically compares current data to baseline values, calculates drift scores using statistical distance metrics such as KL discrepancy, Wasserstein distanceand tracks schema version changes. It also maintains a history of changes, applies significance testing to distinguish real drift from noise, and generates drift reports with severity levels and recommended actions.

Download the data drift detector script

# 4. Validation of hierarchical and graph relations

// Pain point

Hierarchical data must remain acyclic and logically ordered. Cyclic reporting chains, self-referencing bills of material, cyclic taxonomies, and inconsistencies between parents and children break recursive queries and hierarchical aggregations.

// What the script does

Validates chart and tree structures in relational data. Detects circular references in parent-child relationships, ensures hierarchy depth constraints are respected, and validates that directed acyclic graphs (DAGs) remain acyclic. The script also checks for orphaned nodes and disconnected subgraphs, and checks that root nodes and leaf nodes comply with business rules. It also checks the limitations of many-to-many relationships.

// How it works

The script creates a graphical representation of hierarchical relationships, uses cycle detection algorithms to find circular references, and performs depth-first and breadth-first traversals to check structure. It then identifies highly related components in supposedly acyclic graphs, inspects the properties of nodes at each level of the hierarchy, and generates a visual representation of problematic subgraphs with specific violation details.

Download the Hierarchical Relationship Validation Script

# 5. Checking referential integrity in tables

// Pain point

Relational data must maintain referential integrity across all foreign key relationships. Orphaned child records, references to deleted or non-existent parent records, invalid codes, and uncontrolled cascading deletions create hidden dependencies and inconsistencies. These breaches corrupt connections, distort reports, interrupt queries, and ultimately make data unreliable and strenuous to trust.

// What the script does

Checks foreign key relationships and consistency between tables. Detects orphaned records that are missing parent or child references, checks cardinality constraints, and checks composite key uniqueness across tables. It also analyzes the effects of cascading deletes before they occur and identifies circular references in multiple tables. The script works with multiple data files at once to validate relationships.

// How it works

The script loads the master dataset and all related reference tables, checks whether foreign key values ​​exist in parent tables, detects orphaned parent records, and orphaned child records. Checks cardinality rules to ensure one-to-one or one-to-many constraints and validates composite keys spanning multiple columns. The script also generates comprehensive reports showing all affected referential integrity violations, number of rows, and specific foreign key values ​​that have not been validated.

Download the referential integrity check script

# Summary

Advanced data validation goes beyond null and duplicate checking. These five scripts support detect semantic violations, timing anomalies, structural drift, and referential integrity violations that completely miss basic quality controls.

Start with a script that will solve your most essential problem. Configure basic profiles and validation rules for a specific domain. Validate your data pipeline to catch issues at the ingestion stage, not the analysis stage. Configure alert thresholds appropriate to your operate case.

Have fun checking it out!

Priya C’s girlfriend is a software developer and technical writer from India. He likes working at the intersection of mathematics, programming, data analytics and content creation. Her areas of interest and specialization include DevOps, data analytics and natural language processing. She likes reading, writing, coding and coffee! He is currently working on learning and sharing his knowledge with the developer community by writing tutorials, guides, reviews, and more. Bala also creates fascinating resource overviews and coding tutorials.