fast.ai v4 lecture 1
- What do you NOT need to do deep learning? (list 3 things)
- Are neural networks a recent invention?
- What are some of the components of a system that can learn?
- What can a sufficiently large network with one hidden layer learn?
- What is a good way to learn to play baseball?
- What is machine learning? How does it compare to regular programming?
- ML Jargon - list some common ML terms
- What is a positive feedback loop?
- What is the most important thing to do in the top-down approach to learning?
fast.ai v4 lecture 2
- What is the difference between a loss and a metric?
- Does using a validation set guarantee we will not overfit?
- Why transfer learning?
- What is p-value?
- What do we need to call on a pathlib.Path to delete the file?
fast.ai v4 lecture 3
- When is it easier to clean your data - before or after training? Why?
- What are two common problems regarding data your model can encounter in prod?
- Can you collect unbiased data?
- What are the three steps of a careful deployment strategy?
- How to improve your model based on new data?
- What is one of the most challenging things (and why) when deploying ML models?
- When are you at risk of introducing a feedback loop?
- What is a very good way to prevent the introduction of feedback loops?
- Why you should start blogging? (list 3 reasons)
- What to blog about?
- Why should you start each project with creating a baseline?
fast.ai v4 lecture 4
- How do you reshape tensors in PyTorch?
- What is the difference between a metric and a loss?
- How to change a value of a tensor in PyTorch without triggering gradient calculation?
- What is the difference between Gradient Descent and Stochastic Gradient Descent?
- What are the two advantages of presizing, an augmentation method for images?
- Using fastai, what is a good way of evaluating you set up image augmentations correctly?
- How to list the steps (and their configuration) that a DataBlock applies to data?
fast.ai v4 lecture 5
- When can feedback loops occur and what are their consequences?
- Do AI algorithms that are so resilient to errors need human supervision?
- What are 3 examples of unintended consquences of the tech that you build?
- In the Volkswagen diesel cheating case, who was the first person to end up in prison?
- How does bureaucracy assign responsibility and why it is relevant to AI?
- Is ethics culturally dependent?
- What is algorithmic colonialism?
- Can you absolutely trust the data you are gathering? Why?
- Why is it important to understand the issues around the use of metrics to track performance?
- What are our online environments designed to be like?
- What are the potential sources of bias in your data / modelling pipeline?
- What is representation bias?
- Often unethical behavior is driven by management - what can an individual engineer do?
- What is evaluation bias?
- What is historical bias?
- What is measurement bias?
- Does even a bit of diversity help?
- Why does algorithmic bias matter?
- How do we debias our data or ensure it is bias free?
- What is disinformation? (list the three things it involves)
- What is the new form of censorship?
- What is Ethical Risk Sweeping?
- What is Expanding the Ethical Circle?
- What is the Think About the Terrible People exercise about?
- What is the objective of the Closing the Loop: Ethical Feedback & Iteration activity?
- Why having a diverse team makes you much more effective?
- Why do we need policy (list 4 reasons) AND ethical industry behavior (list 2 reasons)?
fast.ai v4 lecture 6
- What is a Dataset in fastai?
- What is Datasets in fastai?
- In Computer Vision, how are the dimensions of an image given?
- In Deep Learning, what can a matrix lookup be replaced with?
- What is the name of the computational shortcut to a matrix lookup done through multiplication?
fast.ai v4 lecture 7
- How does L2 regularization (weight decay) work?
- Using the sigmoid trick with a model that should be able to predict values up to 5 - what max value do you need to set?
- What do we call a variable that can be ordered? (e.g. small, medium, big). How do you convey this to pandas?
- A key technique to modern machine learning goes by the name of bagging. Who invented it? How does it work?
- Does adding more predictors (trees) to Random Forest increase the risk of overfitting?
- What is a partial dependence plot?
- What is the extrapolation problem in the context of Random Forests?
- In the context of a Random Forest, how to find out of domain data?
- What is boosting?
- Can embeddings learned by neural networks be useful to other types of models (i.e. Random Forests or KNN classifiers)?
fast.ai v4 lecture 8
- What is a language model?
- What are the three approaches to tokenization?
- For NLP, is it better to unfreeze many layers at once or gradually unfreeze layers, one by one?
- What is a stateful RNN?
- What is activation regularization (AR)?
- What is temporal activation regularization (TAR)?
- What is weight tying?
fast.ai v3 lecture 8
- What should you focus on when starting to construct a machine learning model?
- What does it mean for a model to overfit?
- What actions should you take when a model overfits?
- How does numpy perform matrix - scalar comparisons?
- What is the name of the procedure that matches the shape of one matrix to another?
- Changing the dimensionality of a matrix
- Which set (train, validation or test) should you use to calculate statistics for normalization?
- As demonstrated in research, what is the trick that enables training a 10_000 layer deep neural network?
- When initializing weights for multiplying relu activations, do you need to take any additional precautions?
- What to be aware of when doing tensor.squeeze()?
fast.ai v3 lecture 9
- What will be the effect of backpropagating the loss multiple times in PyTorch?
- How to take an arbitrary action when setting the value of an attribute on a Python object?
- Which way of calculating the cross entropy loss is faster?
- What issues can arise when running calculations with large floating point numbers?
- What is the benefit of the LogSumExp trick?
- How do iterators behave in Python?
- What does torch.randperm do?
- What do model.train() and model.eval() in PyTorch do?
- What is "proclaim_victory" in the code above?
- How to safely access an attribute that might not exist on a given instance in Python?
- How does __getattr__ work in Python?
- In Python, what is a decorator?
- How does monkey patching work in Python?
fast.ai v3 lecture 10
- Assume you are not keeping up with everything that is covered in the fast.ai course. Should you worry?
- Why are callbacks great for researchers?
- What is a closure?
- How can you set the value of some arguments to a function in Python without calling it?
- What method do you need to define to make a Python object callable?
- What method do you need to define to make a Python object subscriptable?
- How to specify a finalizer (a method that will be called when an instance is being destroyed)?
- What method do you need to define to specify entering the runtime context of an object?
- What is the name of the dunder method returning the length of an object?
- What method should ideally return a string representation of an object that could be used to recreate it?
- What is the method that should compute an "informal" or nicely printable string representation of an object.
- What do you need to define to implement additions between instances of a class?
- What does variance (informally) measure?
- What would taking the mean of the absolute values of differences between values in a tensor and their mean calculate?
- How is standard deviation calculated?
- Which is more sensitive to outliers, mean absolute deviation or standard deviation?
- Why do mathematicians and statisticians use standard deviation more often than mean absolute deviation?
- What is covariance?
- What does the product of differences between values in two tensors and their respective means calculate?
- What is Pearson correlation coefficient?
- When should you and shouldn't you use softmax?
- For classification, what is a good way of handling items in your dataset that don't belong to any of the classes?
- For BatchNorm, what are the normalization statistics calculated on during training?
- If you have a single channel image and are running a 3x3 conv on it, why is opting for 8 channels problematic?
- In a class inheriting from `nn.Module`, what does `self.register_buffer` do?'
- How to calculate moving average with an exponential decay in PyTorch?
- What is a major limitation of BatchNorm?
- How does LayerNorm differ from BatchNorm?
- What are two ways of fixing the small batch problem that affects BatchNorm?
- In the context of exponential moving average, what debiasing or bias correction mean?
fast.ai v3 lecture 11
- What is LSUV (layerwise sequential unit variance)?
- What is a big part of getting good at using deep learning in any given domain?
- What Python tools allow you to work with files / directories in a way that is fast?
- What can you sort using `sorted`?
- How can you handle your model being run at inference on classes it has not seen during training?
- How do you create a class method in Python?
- What are PyTorch hooks?
- Does L2 regularization have a regularizing effect when used with BatchNorm?
- What is LAMB?
- When resizing an image for training, which downsampling algorithm should you use? Nearest or bilinear?
- Are operations faster on Byte or Float tensors?
- What is a fantastically useful data augmentation technique that can be applied across multiple domains?
- In general, does reflection padding help models?
fast.ai v3 lecture 12
- Debugging machine learning code is hard. Does it make sense to follow strict formatting guidelines?
- What is mixup and what are its uses?
- What is label smoothing?
- When training with fp16, are all calculations done using half-precision floats?
- What are the main improvements (over ResNet) implemented in XResNet?
- What is the most important technique to apply to your code, regardless if you are doing research or moving a model to...
- What do you need to call on an `nn.Module` to export its parameters, registered buffers, etc to a dictionary of values?
- What is a very important aspect that is often overlooked when doing transfer learning?
- What are Jeremy Howard's tips for debugging deep learning?
- What is a scientific journal?
- What are the two activation functions used inside the LSTM cell?
- How to split a tensor into some number of pieces of equal size?
- Which state or states does the AWD-LSTM in fast.ai grab to perform classification?