Sponsored content
How much time do you spend on the fight against tools instead of solving problems? Each data scientist was there: down sampling a set of data, because he will not fit in the memory or hacking a way to enable the interaction of a business user with a machine learning model.
The perfect environment disappears, so you can focus on analysis. In this article, it includes eight practical methods in Bigquery, which are supposed to do it exactly, from using AI powered agents to support for ML models straight from the spreadsheet.
1. Machine learning in your spreadsheet
BQML training and forecasts from Google sheet
Many data conversations begin and end in a spreadsheet. They are known, uncomplicated to apply and are great to cooperate. But what happens when your data is too huge for a spreadsheet or when you want to run a forecast without writing code? Connected sheets It helps, enabling the analysis of billions of huge data poems from the Google Sheets interface. All calculations, charts and rotary tables are powered by Bigquery behind the scenes.
Going a step further, you can also access the models you have built Bigquery Machine Learning (BQML). Imagine you have a BQML model that predicts the prices of apartments. Thanks to the connected sheets, a business user can open a sheet, enter data for a modern property (square material, bedroom number, location), and the formula can call the BQML model to return the price estimation. None needed Python or API Wranling – only the sheet formula calls the model. This is a powerful way to expose machine learning to non -technical teams.
2
The first steps with Enterprise data warehouses are often associated with friction, such as setting up a billing account. . Bigquery Sandbox Removes this barrier, allowing you to ask for 1 terabajt data per month. Does not require a credit card. This is great, without valuables to start learning and experimenting with huge -scale analyzes.
As a data scientist, you can access the Bigquery sandbox with Colab notebook. Thanks to only a few lines of authentication code, you can start SQL queries directly from the notebook and download the results for python data analysis. The same notebook environment can even act as an AI partner to assist plan the analysis and the recording code.
3. Your AI powered partner in Colab notebooks
Agent Data Science in Colab notebook (low sequences, results for illustration purposes)
Colab notebooks are now First experience Designed to accelerate workflow. You can generate a natural language code, get automatic errors explanations and talk to the assistant next to the code.
Colab notebooks also have a built -in data science agent. Think about how you can work about the ML expert. Start with a set of data – such as the local CSV or huge table – and a high level target, for example “build a model of predicting the client’s departure”. The agent creates a plan with suggested steps (e.g. data cleaning, function engineering, model training) and saves the code.
And you are always under control. The agent generates the code directly in notebook cells, but does not start anything himself. You can review and edit any cell before deciding what to do, and even ask the agent to rethink his approach and try various techniques.
4
Many data scientists live in notebooks and apply men data to manipulate data. But there is a well -known limit: all the data you process must fit in the device’s memory. MemoryError Exceptions are too common, which forces you to lose data early.
This is a exact problem Bigquery Dataframe Solves. Provides the API Python interface, intentionally similar to Panda. Instead of acting locally, he explains your commands on SQL and performs them in a bigquery engine. This means that you can work with Terabyte data sets from the notebook, with a known API interface and without worrying about memory limitations. The same concept applies to model training, with the API interface similar to Scikit-Learn, which pushes model training to Bigquery ML.
5. Spark ML in Bigquery Studio Notebooks
Examples of Spark ML notebook in Bigquery Studio
Apache Spark is a useful tool, from function engineering to model training, but infrastructure management has always been a challenge. Serverless for Apache Spark It allows you to launch the spark code, including tasks using libraries such as XGBOOST, Pytorch and Transformers, without having to provide a cluster. You can develop interactive From the notebook directly in Bigquery, allowing you to focus on the development of models, and Bigquery supports infrastructure.
You can apply Serverless Spark to work on the same data (and thus the management model) in the Bigquery magazine.
6. Add an external context to public data sets
5 best terms in the Los Angeles region at the beginning of July 2025.
Your first page data tell you what happened, but I can’t always explain why. To find this context, you can join your data with a huge collection of public data sets available in Bigquery.
Imagine you are a scientist of the retail brand data. You see an escalate in the sales of the raincoat in the north -western Pacific. Was it your last marketing campaign or something else? Attaching to the sales data from Google Trends data set In Bigquery, you can quickly check whether the “waterproof jacket” also increased in the same region and the period.
Or let’s say you are planning a modern store. You can apply Places the Insights Data set To analyze movement patterns and business density in potential districts, applying them in addition to customer information to choose the best location. These public data sets allow you to build richer models that include real factors.
7. Georphagen analytics on a scale
Bigquery Geo Map of Hurricane, using the color to indicate the radius and wind speed
Building the function of the model can be convoluted, but Bigquery simplifies it through support GEOGRAPHY Data type and standard GIS functions within SQL. This allows spatial function engineers directly at the source. For example, if you are building a model of anticipating real estate prices, you can apply a function such as ST_DWithin to calculate the number of public transport stops within one mile radius for each property. Then you can apply this value directly as the entrance to the model.
You can go on Google Earth Engine Integration, which introduces petabytes of satellite photos and environmental data to Bigquery. In the case of the same real estate model, you can ask about the data of the Earth’s engine to add functions such as historical flood risk and even the density of the tree covering. This helps build much richer models by increasing business data about information about the environment on the planet’s scale.
8. Reason Data on the Journal
Most people think about Bigquery for analytical data, but it is also a powerful destination destination. You can manage your whole thing Bigquery cloud registration dataTransforming unstructured text diaries into asking resources. This allows SQL to be launched between diaries of all your services to diagnose problems, track performance or analyze security events.
For the scientist, data registration data in the cloud is a prosperous source for building forecasts. Imagine that you are investigating a decrease in user activity. After identifying the error message in the diaries, you can apply Bigquery vectors search To find semantically similar diaries, even if they do not contain exactly the same text. This can assist to reveal related problems, such as “abnormal user token” and “authentication failed”, which are part of the same reason. Then you can apply these marked data to train the anomalies detecting model, which inactively means patterns.
Application
We hope that these examples stimulate modern ideas for the next project. From pandas dataframe to engineering with geographical data, the goal is to assist in working with known tools.
Ready to try? You can start discovering without any costs in Bigquery Sandbox!
Author: Jeff Nelson, relationship engineer with programmers