Photo by the editor
# Entry
# 1. Neural networks and deep learning
Bible Neural networks and deep learning takes you from the basics of neural networks to building and training deep models yourself. It starts with plain ideas like perceptrons and sigmoid neurons, and then walks you through creating a network that can recognize handwritten digits. You can also see how backpropagation actually works when training these models and how to improve them with things like cost functions, regularization, weight initialization, and hyperparameter tuning. There are lots of Python code examples, so you can test things out for yourself and see how it all comes together. It combines intuition and mathematics nicely, so you start to understand more than just that How neural networks work, but Why. If you already know some math (e.g. linear algebra or calculus), this is a good choice if you want to go beyond just using the library and learn what’s going on under the hood.
// Outline overview:
- Basics of neural networks (Perceptrons, sigmoid neurons, network architecture, handwritten digits classification, gradient descent, implementing networks)
- Backpropagation and learning (Matrix calculations, cost function assumptions, Hadamard product, four basic backpropagation equations, algorithm implementation, learning improvement)
- Advanced training techniques (Cross entropy cost, overfitting and regularization, weight initialization, hyperparameter selection, universality of neural networks, extensions beyond sigmoid neurons)
- Deep learning and convolutional networks (Vanishing Gradient Problem, Unstable Gradients, Convolutional Neural Networks, Practical Implementations, Recent Advances in Image Recognition, Future Directions)
# 2. Deep learning
Deep learning gives a really good overview of deep learning and how machines actually learn from experience, building convoluted ideas from simpler ones. It starts with the math you need, like linear algebra, probability, information theory, and a bit of numerical computing, and then moves on to the basics of machine learning. We then dive deeper into state-of-the-art deep learning methods such as feedforward, convolutional and recurrent networks, regularization and optimization, showing how they are used in real-world projects. It also covers some advanced topics such as autoencoders, generative and representational learning, and structured probabilistic models. It is intended primarily for people with a solid understanding of mathematics, so it reads more like a suitable textbook for research or advanced work than a beginner’s guide.
// Outline overview:
- Factor models and autoencoders (PCA, ICA, limited coding, incomplete and regularized autoencoders, denoising, multiple learning)
- Representation learning and probabilistic models (Layered pre-learning, transfer learning, distributed representations, structured probabilistic models, approximate inference, Monte Carlo methods)
- Deep generative models and advanced techniques (Boltzmann machines, deep belief networks, convolutional models, generative stochastic networks, autoencoder sampling, evaluation of generative models)
# 3. Practical deep learning
To combine:
Free course Practical deep learning is designed for people who already know some coding and want to gain practical skills in machine learning and deep learning. Instead of just reading theory, you’ll immediately start building models for real-world tasks. The course covers state-of-the-art tools such as Python, PyTorchand Quick Library and shows you how to handle everything from data cleansing to model training, testing, and deployment. You’ll work with real notebooks, data sets and problems, so you learn by doing. The emphasis is on practical, up-to-date methods for selecting the appropriate algorithm, its proper validation, scaling and implementation.
// Outline overview:
- Basics and model training (Basics of neural networks, stochastic gradient, affine functions and nonlinearities, back propagation, MLP, autoencoders)
- Applications in different domains (Computer vision in CNNs, natural language processing (NLP) including phrase embedding and similarity, tabular data modeling, collaborative filtering and recommendations)
- Advanced techniques and optimization (Transfer learning, mass decay, data augmentation, accelerated stochastic gradient descent (SGD), ResNets, mixed precision, DDPM/DDIM, attention and transformers, latent diffusion, super resolution)
- Implementation and practical skills (Transforming models into web applications, improving accuracy/speed/reliability, ethical considerations, frameworks like The Learner, matrix operations, model initialization/normalization)
# 4. Artificial intelligence: basics of computational agents
Bible Artificial intelligence: the basics of computational agents explains artificial intelligence through the concept of “computational agents,” systems that can sense, learn, reason, and act. The latest edition adds newer topics such as neural networks, deep learning, causality, and the social and ethical sides of artificial intelligence. It shows how agents are built, how they plan and act, and how they deal with convoluted or uncertain situations. Each chapter contains algorithms in Pythoncase studies and real-world discussions so you learn both the how and the why. It’s a balanced combination of theory and practice, perfect for students and anyone who wants a state-of-the-art and deep introduction to artificial intelligence.
// Outline overview:
- Basics of artificial intelligence and agents (natural vs. artificial intelligence, historical context, agent design space, and examples such as delivery robots, diagnostic assistants, learning systems, sales agents, and clever homes).
- Agent architectures and control (hierarchical control, agent functions, offline and online computation, and how agents perceive and act in environments).
- Reasoning, planning and searching (solving problems through search, graph traversal, constraint satisfaction, probabilistic reasoning and planning methods including forward planning, regression and partial order planning)
- Learning and neural networks (supervised learning, decision trees, regression, overfitting, convoluted models such as reinforcement, deep learning architectures (convolutional neural networks (CNN), recurrent neural networks (RNN), transformers) and immense language models).
- Uncertainty, causality, and reinforcement learning (probabilistic reasoning, Bayesian learning, unsupervised methods, causal inference, decision making under uncertainty, sequential decisions and reinforcement learning strategies such as Q-learning and evolutionary algorithms).
# 5. Ethical artificial intelligence
Paper Ethical artificial intelligence examines how future artificial intelligence systems may behave in ways we don’t expect or that could be harmful, and suggests ways to design them safely. It starts by pointing out that AI can learn models of the world that are much more convoluted than humans can fully understand, making it arduous to apply security. The authors recommend using utility functions (mathematical descriptions of what the AI should be interested in) rather than vague rules because this makes the goals clearer. It also includes issues like self-deception, where the AI can distort its own observations or rewards, unintentional shortcuts that hurt us, and reward generator corruption, where the AI manipulates its own reward system. The authors propose models that learn human values, employ finite definitions, and include self-modeling so that artificial intelligence can draw conclusions about its own actions. It also takes into account the bigger picture, such as the impact of artificial intelligence on society, politics and the future of humanity.
// Outline overview:
- AI foundations and design (future AI vs. current AI, instructing AI, utility maximizing agents, learning environment models, intelligence measures, ethical frameworks)
- AI behavior and challenges (self-deception, unintended instrumental actions, model-based utility functions, human value learning, evolving and embedded factors)
- Testing, Governance and Society (AI testing, real-world behavior, political dimensions, transparency, allocation of benefits, ethical considerations)
- Philosophical and social influence (search for meaning, social and cultural implications, linking computation and human values)
# Summary
These books (and the article and course) cover a wide range of needs as an AI engineer, from neural networks and deep learning to hands-on coding, agent-based AI, and ethical issues. They provide a clear path from exploring ideas to applying artificial intelligence in real situations. What topics would you like me to cover next? Post your suggestions in the comments!
Kanwal Mehreen is a machine learning engineer and technical writer with a deep passion for data science and the intersection of artificial intelligence and medicine. She is co-author of the e-book “Maximizing Productivity with ChatGPT”. As a 2022 Google Generation Scholar for APAC, she promotes diversity and academic excellence. She is also recognized as a Teradata Diversity in Tech Scholar, a Mitacs Globalink Research Scholar, and a Harvard WeCode Scholar. Kanwal is a staunch advocate for change and founded FEMCodes to empower women in STEM fields.