MLOps Zoomcamp 2024

Homework 4: Deployment Statistics

Distribution of scores and reported study time for this homework.

Submissions

298

Median total score

6

Average total score

6

Score distribution

All values are points.

Questions score

Min
2
Median
6.0
Max
6
Q1
5.0
Avg
5.4
Q3
6.0

Learning in public score

Min
-
Median
0.0
Max
7
Q1
0.0
Avg
0.7
Q3
1.0

Total score

Min
2
Median
6.0
Max
14
Q1
5.0
Avg
6.3
Q3
7.0

Time distribution

All values are hours reported by students.

Lectures

Min
1.0
Median
4.0
Max
60.0
Q1
2.5
Avg
5.4
Q3
6.0

Homework

Min
0.5
Median
3.0
Max
16.0
Q1
2.0
Avg
3.7
Q3
5.0

Question breakdown

Correctness and answer distribution per question.

1. Notebook

285 / 298 correct (95.6%)

1 1.24 3 (1.0%)
2 6.24 285 (95.6%)
3 12.28 6 (2.0%)
4 18.28 3 (1.0%)

2. Preparing the output

272 / 298 correct (91.3%)

1 36M 5 (1.7%)
2 46M 7 (2.3%)
3 56M 11 (3.7%)
4 66M 272 (91.3%)

3. Creating the scoring script

298 / 298 correct (100.0%)

Answer Count
jupyter nbconvert --to script starter.ipynb 117
jupyter nbconvert --to script score.ipynb 20
!jupyter nbconvert --to script starter.ipynb 11
jupyter nbconvert --to python starter.ipynb 10
jupyter nbconvert --to script homework.ipynb 6
jupyter nbconvert --to=script starter.ipynb 5
jupyter nbconvert --to script 5
jupyter nbconvert 4
nbconvert 4
jupyter nbconvert --to script filename.ipynb 3
jupyter nbconvert --to script homework4.ipynb 3
jupyter nbconvert --to script scoring.ipynb 3
!jupyter nbconvert --to script --output-dir . starter.ipynb 3
jupyter nbconvert --to script notebook.ipynb 3
python script.py 2
python predict.py 2
jupyter nbconvert --to script your_notebook.ipynb 2
jupyter nbconvert --to python notebook.ipynb 2
jupyter nbconvert --to script nazmul_homework_4.ipynb --output script 1
jupyter nbconvert --to script .\starter.ipynb 1
jupyter nbconvert --to script file_name.ipynb 1
jupyter nbconvert --to script start.ipynb 1
jupyter nbconvert --to script predict.ipynb 1
cp .. 1
jupyter nbconvert –to script score_2_script_clean.ipynb 1
jupyter nbconvert --to script notebook_name.ipynb 1
jupyter nbconvert --to script duration_pred.ipynb 1
jupyter nbconvert --to script /path/to/notebook.ipynb 1
!jupyter nbconvert --to script ./homework/starter.ipynb 1
jupyter nbconvert --to script Home_work_Module_4.ipynb 1
jupyter nbconvert -to=script starter.ipynb 1
jupyter nbconvert homework/starter.ipynb --to python --output prediction_script.py 1
jupyter nbconvert starter.ipynb --to python 1
jupyter nbconvert --to script notebook.ipynb --output predict 1
python name0fTheScript.py 2003 04 1
jupyter nbconvert --to script HW_4.ipynb 1
Jupyter nbconvert --to script <notebook name> 1
jupyter nbconvert -- to script 1
jupyter nbconvert --to script your_name_file.ipynb 1
jupyter nbconvert --to script "path/to/notebook.ipynb" 1
jupyter nbconvert --to python Untitled.ipynb 1
jupyter nbconvert --to python homework-04-deployment.ipynb 1
jupyter nbconvert --to script hw_adi_04-deploy.ipynb 1
!jupyter nbconvert --to script ./description_homework/starter.ipynb --output ../trans_from_starter_ipynb 1
jupyter-nbconvert starter.ipynb --to script 1
jupyter nbconvert notebook.ipynb --to script 1
jupyter nbconvert --to script --output my_script.py notebook.ipynb 1
jupyter-nbconvert --to script starter.ipynb 1
jupyter nbconvert --to script {starter}.ipynb 1
jupyter nbconvert --to script <notebook file name> 1
File -> Save and export notebook as -> executable script 1
jupyter nbconvert --to script W04_HW.ipynb 1
jupyter nbconvert --to script model_scorer.ipynb 1
jupyter nbconvert homework.ipynb --to python 1
jupyter nbconvert --to script nomeNotebook.ipynb 1
jupyyter nbconvert --to script starter.ipynb 1
jupyter nbconvert --to script starter.ipynb --no-prompt --output "predict" 1
juoter nbconvert --to script starter.ipynb 1
jupyter nbconvert --to script ./00-homework-solns/04-deployment.ipynb 1
jupyter nbconvert --to script name_of_file.ipynb 1
jupyter nbconvert –to script starter.ipynb 1
jupyter nbconvert --to script score.py 1
jupyter nbconvert --to script homework-04-score.ipynb 1
python score.py yellow 2023 3 1
jupyter nbconvert --to script 04-homework.ipynb 1
jupyter nbconvert --to script homework/starter.ipynb 1
jupyter nbconvert --to script MLOps_Homework_4.ipynb 1
jupyter nbcconvert --to script score.ipynb 1
jupyter nbconvert --to script [YOUR_NOTEBOOK].ipynb 1
!jupyter nbconvert --to script hw4.ipynb 1
jupyter nbconvert --to script --output scoring starter.ipynb 1
use the Command Palette in VSCode Press Ctrl+Shift+P (Windows/Linux) or Cmd+Shift+P (Mac) to open the Command Palette. Type "Export" Choose "Export to Python Script" A python file will pop up Save the file 1
jupyter nbconvert my_solution.ipynb --to python 1
!jupyter nbconvert --to script --no-prompt homework.ipynb --TemplateExporter.exclude_markdown=True --TemplateExporter.exclude_output_prompt=True --TemplateExporter.exclude_input_prompt=True 1
jupyter nbconvert --to script deploy_homework.ipynb 1
python predict.py --year 2023 --month 4 1
!jupyter nbconvert --to script homework-04-starter-notebook-Copy1.ipynb 1
jupyter nbconvert --to script 04-deployment.ipynb 1
jupyter nbconvert --to script Module_4_Homework.ipynb 1
scoring.py 1
!jupyter nbconvert --to python starter.ipynb --PythonExporter.exclude_markdown=True --PythonExporter.exclude_input_prompt=True --RegexRemovePreprocessor.patterns="^[!%]" 1
python starter.py 1
jupyter nbconvert --to script ./starter.ipynb 1
python3 main.py 1
jupyter nbconvert --to script homework-04.ipynb 1
jupyter nbconvert --to script 04_deployment.ipynb 1
!upyter nbconvert --to script homework.ipynb --output score 1
#!jupyter nbconvert --to script starter.ipynb 1
jupyter nbconvert --to python mlops_zoomcamp_homework_4.ipynb 1
jupyter nbconvert starter.ipynb --to script 1
!jupyter nbconvert --to script homework_04_deployment.ipynb 1
jupyter nbconvert --to python score.ipynb 1
jupyter nbconvert --to script starter_ag.ipynb 1
jupyter nbconvert --to script <filename_to_convert> 1
jupyter nbconvert --to script starter.py 1
jupyter nbconvert --to script score_.ipynb 1
jupyter nbconvert --to script Scoring.ipynb 1
jupyer nbconvert --to script score.ipynb 1
jupyter nbconvert --to script homework_notebook.ipynb 1
jupyter nbconvert --to script <notebook_name>.ipynb 1
jupyter nbconvert --to script homework_04.ipynb 1
#!/usr/bin/env python # coding: utf-8 import pickle import pandas as pd import sys import s3fs import os categorical = ['PULocationID', 'DOLocationID'] def load_model(path='model.bin'): with open(path, 'rb') as f_in: dv, model = pickle.load(f_in) return dv, model def read_data(filename: str): df = pd.read_parquet(filename) df['duration'] = df.tpep_dropoff_datetime - df.tpep_pickup_datetime df['duration'] = df.duration.dt.total_seconds() / 60 df = df[(df.duration >= 1) & (df.duration <= 60)].copy() df[categorical] = df[categorical].fillna(-1).astype('int').astype('str') return df def make_predictions(df, dv, model): dicts = df[categorical].to_dict(orient='records') X_val = dv.transform(dicts) y_pred = model.predict(X_val) return y_pred #Create an aritificial column for ride_id def create_ride_ids(df, year, month): df['ride_id'] = f'{year:04d}/{month:02d}_' + df.index.astype('str') return df def save_results(df, y_pred, output_file): #Write the ride_id and the predictions to a dataframe with the results df_result = pd.DataFrame({ 'ride_id': df['ride_id'], 'prediction': y_pred }) #Save it as a parquet df_result.to_parquet( output_file, engine='pyarrow', compression=None, index=False ) def run(): #Take the year and month from the arguements year = int(sys.argv[1]) month = int(sys.argv[2]) bucket_name = str(sys.argv[3]) df = read_data(f"https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_{year}-{month:02}.parquet") dv, model = load_model() y_pred = make_predictions(df, dv, model) #Std deviation of Duration y_pred_series = pd.Series(y_pred) std_deviation = y_pred_series.std() print("Standard Deviation of Predictions:", std_deviation) # Calculate the mean of predictions mean_of_predictions = y_pred_series.mean() print("Mean of Predictions:", mean_of_predictions) # Create an artificial ride_id column df = create_ride_ids(df, year, month) output_file = f's3://{bucket_name}/nyc-taxi-duration/results.parquet' save_results(df, y_pred, output_file) if __name__ == '__main__': run() 1
$ jupyter nbconvert --to script homework_4.ipynb 1
jupyter nbconvert homeworks/04-deployment/starter.ipynb --no-prompt --to python 1

4. Virtual environment. Hash for Scikit-Learn

298 / 298 correct (100.0%)

Answer Count
sha256:057b991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c 132
057b991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c 39
"sha256:057b991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c" 22
sha256:08ef968f6b72033c16c479c966bf37ccd49b06ea91b765e1cc27afefe723920b 20
sha256:1d0b25d9c651fd050555aadd57431b53d4cf664e749069da77f3d52c5ad14b3b 11
"sha256:08ef968f6b72033c16c479c966bf37ccd49b06ea91b765e1cc27afefe723920b" 5
08ef968f6b72033c16c479c966bf37ccd49b06ea91b765e1cc27afefe723920b 5
1d0b25d9c651fd050555aadd57431b53d4cf664e749069da77f3d52c5ad14b3b 3
"sha256:0402638c9a7c219ee52c94cbebc8fcb5eb9fe9c773717965c1f4185588ad3107" 2
sha256:065e9673e24e0dc5113e2dd2b4ca30c9d8aa2fa90f4c0597241c93b63130d233 2
pipenv install scikit-learn==1.5.0 pyarrow pandas --python=3.10 2
"sha256:057b991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c", 2
sha256:0df87de9ce1c0140f2818beef310fb2e2afdc1e66fc9ad587965577f17733649 2
sha256:097b0ff11cd452f01099e7386c8e45bc2d614da5a7bb8bc8daef5fdf8bf1fe83 1
sha256:12e40ac48555e6b551f0a0a5743cc94cc5a765c9513fe708e01f0aa001da2801 1
a015d3efe592085b07d302c00f81ae954e6fec0b0401cfccc55a383401a446be 1
69f4387a511f49e2575f793043bc7e8ce82df97c18b72a691b20493c5b2a0662 1
sha256:0e8102d5036e28d08ab47166b48c8d5e5810704daecf3a476a4282d562be9a28 1
BA 1
sha256:05fc5915b716c6cc60a438c250108e9a9445b522975ed37e416d5ea4f9a63381 1
"hash": { "sha256": "a7e186ee6687fadf743e6641590d61c720841707bc17859c1d48df95a04f1e56"} 1
first has: "sha256:057b991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c" 1
065e9673e24e0dc5113e2dd2b4ca30c9d8aa2fa90f4c0597241c93b63130d233 1
8ef968f6b72033c16c479c966bf37ccd49b06ea91b765e1cc27afefe723920b 1
"sha256:0e8102d5036e28d08ab47166b48c8d5e5810704daecf3a476a4282d562be9a28" 1
sha256:1234567890abcdef... 1
38fe20c5895c057cdb6fb8a599681540ab914ccf573e3b70a73dcb509c1d7cde 1
scikit-learn=1.5.0 1
sha256:0a127cc70990d4c15b1019680bfedc7fec6c23d14d3719fdf9b64b22d37cdeca 1
22914eb0db878c39d1a8c0ec4b17ff31b448fa1bb82b321889ef7b00b8f1e0d0 1
32d3a61add28d550847add3defec94703988dc41ce1b61e1d8aef6fe57ce4317 1
"scikit-learn": { "hashes": [ "sha256:08ef968f6b72033c16c479c966bf37ccd49b06ea91b765e1cc27afefe723920b", "sha256:158faf30684c92a78e12da19c73feff9641a928a8024b4fa5ec11d583f3d8a87", "sha256:16455ace947d8d9e5391435c2977178d0ff03a261571e67f627c8fee0f9d431a", "sha256:245c9b5a67445f6f044411e16a93a554edc1efdcce94d3fc0bc6a4b9ac30b752", 1
"sha256": "c8122227c10016a12c68f3bb3d4e50d0704b0ccf8f0dfa9a97195d4fcf7cdb14" 1
ad8a8c1078161eba6e9d3de09f71d393d822592a7abe46fcdd22480a3b0dfb3f 1
!pipenv install scikit-learn==1.5.0 pandas --python=3.10 1
sha256:118a8d229a41158c9f90093e46b3737120a165181a1b58c03461447aa4657415 1
sha256:038f4e9d6ef10e1f3fe82addc3a14735c299866eb10f2c77c090410904828312 1
- 1
7c23eb0b4a2e660a4baf6f5f9c117b75c857d9d1e9b5a9e2924a7693c1c4d05b 1
scikit-learn = "==1.2.2" 1
"sha256:065e9673e24e0dc5113e2dd2b4ca30c9d8aa2fa90f4c0597241c93b63130d233" 1
"sha256": "c5b92ca78834b33a1d33e21a0e4f307718034a6d801e22a4749935cfce0f40fb" 1
# scikit-learn hash # 057b991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c 1
673dbd3dbb53d711fba58ae14349c0a10cf9557bcac286f2a3a420546d3b44b1 1
sha256:0e8102d5036e28d08ab47166b48c8d5e5810704daecf3a476a4282d562be9a2 1
991ac64b3e75c9c04b5f9395eaf19a6179244c089afdebaad98264bff37c 1
pipenv install scikit learn 1
"sha256:daa1c471d95bad080c6e44b4946c9390a4842adc3082572c20e4f8884e39e959" 1
99ea61d9145ebd976fe45f877b7f2e3624e828b6343d0c48978d66910a997b91 1
"scikit-learn": { "hashes": [ "sha256:08ef968f6b72033c16c479c966bf37ccd49b06ea91b765e1cc27afefe723920b", "sha256:158faf30684c92a78e12da19c73feff9641a928a8024b4fa5ec11d583f3d8a87", "sha256:16455ace947d8d9e5391435c2977178d0ff03a261571e67f627c8fee0f9d431a", "sha256:245c9b5a67445f6f044411e16a93a554edc1efdcce94d3fc0bc6a4b9ac30b752", "sha256:285db0352e635b9e3392b0b426bc48c3b485512d3b4ac3c7a44ec2a2ba061e66", "sha256:2f3b453e0b149898577e301d27e098dfe1a36943f7bb0ad704d1e548efc3b448", "sha256:46f431ec59dead665e1370314dbebc99ead05e1c0a9df42f22d6a0e00044820f", "sha256:55f2f3a8414e14fbee03782f9fe16cca0f141d639d2b1c1a36779fa069e1db57", "sha256:5cb33fe1dc6f73dc19e67b264dbb5dde2a0539b986435fdd78ed978c14654830", "sha256:75307d9ea39236cad7eea87143155eea24d48f93f3a2f9389c817f7019f00705", "sha256:7626a34eabbf370a638f32d1a3ad50526844ba58d63e3ab81ba91e2a7c6d037e", "sha256:7a93c1292799620df90348800d5ac06f3794c1316ca247525fa31169f6d25855", "sha256:7d6b2475f1c23a698b48515217eb26b45a6598c7b1840ba23b3c5acece658dbb", "sha256:80095a1e4b93bd33261ef03b9bc86d6db649f988ea4dbcf7110d0cded8d7213d", "sha256:85260fb430b795d806251dd3bb05e6f48cdc777ac31f2bcf2bc8bbed3270a8f5", "sha256:9369b030e155f8188743eb4893ac17a27f81d28a884af460870c7c072f114243", "sha256:a053a6a527c87c5c4fa7bf1ab2556fa16d8345cf99b6c5a19030a4a7cd8fd2c0", "sha256:a90b60048f9ffdd962d2ad2fb16367a87ac34d76e02550968719eb7b5716fd10", "sha256:a999c9f02ff9570c783069f1074f06fe7386ec65b84c983db5aeb8144356a355", "sha256:b1391d1a6e2268485a63c3073111fe3ba6ec5145fc957481cfd0652be571226d", "sha256:b54a62c6e318ddbfa7d22c383466d38d2ee770ebdb5ddb668d56a099f6eaf75f", "sha256:b5870959a5484b614f26d31ca4c17524b1b0317522199dc985c3b4256e030767", "sha256:bc3744dabc56b50bec73624aeca02e0def06b03cb287de26836e730659c5d29c", "sha256:d93d4c28370aea8a7cbf6015e8a669cd5d69f856cc2aa44e7a590fb805bb5583", "sha256:d9aac97e57c196206179f674f09bc6bffcd0284e2ba95b7fe0b402ac3f986023", "sha256:da3c84694ff693b5b3194d8752ccf935a665b8b5edc33a283122f4273ca3e687", "sha256:e174242caecb11e4abf169342641778f68e1bfaba80cd18acd6bc84286b9a534", "sha256:eabceab574f471de0b0eb3f2ecf2eee9f10b3106570481d007ed1c84ebf6d6a1", "sha256:f14517e174bd7332f1cca2c959e704696a5e0ba246eb8763e6c24876d8710049", "sha256:fa38a1b9b38ae1fad2863eff5e0d69608567453fdfc850c992e6e47eb764e846", "sha256:ff3fa8ea0e09e38677762afc6e14cad77b5e125b0ea70c9bba1992f02c93b028", "sha256:ff746a69ff2ef25f62b36338c615dd15954ddc3ab8e73530237dd73235e76d62" ], 1
sha256:23fb9e74b813cc2528b5167d82ed08950b11106ccf50297161875e45152fb311 1

5. Parametrize the script

268 / 298 correct (89.9%)

1 7.29 10 (3.4%)
2 14.29 268 (89.9%)
3 21.29 12 (4.0%)
4 28.29 3 (1.0%)

6. Docker container

196 / 298 correct (65.8%)

1 0.19 196 (65.8%)
2 7.24 13 (4.4%)
3 14.24 67 (22.5%)
4 21.19 12 (4.0%)

Calculated: 13 October 2024, 05:00