6.4. Formative quiz 6

## Re-arrange the below into the approximate order for creating a new AI model. 1. Defining a problem statement 2. Getting data 3. Pre-processing data 4. Training the model 5. Testing the model 6. Generating performance metrics ## What is the name of the Python module we used for creating AI models? 1. [x] PyTorch > That's correct! 1. [ ] TensorFlow > That's not correct. TensorFlow is also used for creating AI models, but it's not what we used. 1. [ ] Keras > That's not correct. Keras is also used for creating AI models, but it's not what we used. 1. [ ] scikit-learn > That's not correct. Scikit-learn is also used for creating AI models, but it's not what we used. 1. [ ] Jax > That's not correct. Jax is also used for creating AI models, but it's not what we used. ## When creating AI models, what sub-data sets are the data typically split into? Select all that apply. - [x] Training > That's correct! - [x] Validation > That's correct! - [x] Testing > That's correct! - [ ] Model > That's not correct. This isn't a standard term used for splitting data sets. - [ ] Analysis > That's not correct. This isn't a standard term used for splitting data sets. ## In a two class classification problem, the performance is summarized in a confusion matrix, shown below.  <br />The classification accuracy is defined as <br /> ` TP + TN ` <br /> `-----------------` <br /> `TP + TN + FP + FN` <br /> where TP is true positives, TN is true negatives, FP is false positives and FN is false negatives. What is the accuracy of this model? > This isn't particularly a question about Python, we're just checking you're thinking about how you would evaluate an AI model. Checking the performance is an extremely important step in creating AI models, and can be quite hard. 1. [x] 88.7% 1. [ ] 33.8% 1. [ ] 66.2% 1. [ ] 100% 1. [ ] 15.4% ## You ask an AI model to classify images of cats and dogs. There are 9 images of cats, and 1 image of a dog. The model classifies all images as cats. What is the accuracy of the model? > Again this isn't particularly a question about Python, we're just checking you're think about how you would evaluate an AI model. If your classes are imbalanced (i.e. one class is much more common than another), accuracy can be a misleading metric to use. If you have a model that just says everything is a cat, it still gets a very good numerical performance. You'll need to think more about the performance evaluation approach. 1. [ ] 0% 1. [ ] 10% 1. [ ] 50% 1. [x] 90% 1. [ ] 100% ## Which of the below are questions that you might like to ask before incorporating AI generated code suggestions? Select all that apply. - [x] Is the code what you want? > That's correct! - [x] Is the code correct? > That's correct! - [x] Is the code the best way to do what I want? > That's correct! - [x] Is use of AI code in-line with relevant permissions and policies? > That's correct! ## A customized prompt for generating suggestions is generally known as: 1. [x] An agent > That's correct! 1. [ ] A bot > That's not correct. A bot is an automated program, but it's not the term used for a customized prompt. 1. [ ] A helper > That's not correct. A helper is a general term, but it's not the term used for a customized prompt. 1. [ ] A script > That's not correct. A script is a piece of code, but it's not the term used for a customized prompt. 1. [ ] A program > That's not correct. A program is a general term, but it's not the term used for a customized prompt. ## AI code has been generated by the computer and so must be correct. 1. [ ] True 1. [x] False ## What uses might AI have in programming, beyond generating code? Select all that apply. - [x] Writing documentation > That's correct! - [x] Helping debugging > That's correct! - [x] Explaining code > That's correct! - [x] Generating test inputs/outputs > That's correct! - [ ] Generating requirements specifications > We've put this as not correct, although there could be some debate on this one. AI could potentially help with this, particularly for generating initial drafts. However it's important that you know and understand the specification and what you're trying to make. If you don't know what you're trying to do, it's extremely hard to tell whether your code is actually doing it or not! ## AI tools will be available in the closed book exam? 1. [ ] True 1. [x] False