Table of contents
- Experiment 3: TensorFlow macOS code
- 3.1 Basic Convolutional Neural Network (CNN)
- 3.2 Transfer learning using EfficientNetB0
- 3.3 tensorflow_macos GitHub benchmark
- TensorFlow code result
- portability
- 16-inch MacBook Pro sold off
- Conclusion
Experiment 3: TensorFlow macOS code
In its M1 chip launch talk, Apple claimed that the new silicon “can run popular deep learning frameworks like TensorFlow much faster than previous generation Macs.”
Hearing that, I had to sit up straight.
Did I just say TensorFlow? And native?
I reviewed the announcement .
Oh, it certainly is. It said TensorFlow.
Then I found a blog post from the TensorFlow team and the Apple Machine Learning team presenting new results on the M1 chip and Intel-based Macs. Apple recently released a fork of TensorFlow, tensorflow_macos, which allows you to run native TensorFlow code on your Mac (running TensorFlow code on your Mac used to be a hassle , but PlaidML * (I haven’t tried anything that I’ve heard makes it easier).
Comparison of AI model training time for various MacBook Pros
In the above graph, the measurement environment was prepared by running TensorFlow 2.3 and TensorFlow 2.4 on a 13-inch MacBook Pro (2020) with an Intel chip, and TensorFlow 2.4 on a 13-inch MacBook Pro (2020) with an M1 chip. As models to be trained, we prepared fine tuning using ResNet50V2, CycleGAN, Style Transfer, MobileNetV3, and DenseNet121.
As you can see from the graph, the MacBook Pro with the M1 chip completed the training the fastest . Training time is indicated by the length of the bar, shorter means faster.
Naturally, upon hearing this news, I had to give it a try.
By some miracle, I installed Apple’s fork of TensorFlow in a Python 3.8 environment within 8-10 hours without troubleshooting and created a mini-experiment like this:
3.1 Basic Convolutional Neural Network (CNN)
I copied the CNN architecture from the CNN explanation site (TinyVGG). And I used data similar to the tests I did with CreateML.
- Challenge: Multi-class image classification.
- Model: TinyVGG (see Google Colab notebook below for exact code).
- Data: 750 training images, 2500 test images.
- Number of classes: 10 (taken from the Food101 dataset).
- Number of epochs: 5
- Batch size: 32
The code used for test 3.1 is a copy of TinyVGG CNN posted on the CNN commentary site with TensorFlow.
3.2 Transfer learning using EfficientNetB0
These days, it’s rare to build a model from scratch. Either use an existing untrained architecture and train on your own data, or use a pre-trained architecture like EfficientNet and fine-tune on your data .
- Challenge: Multi-class image classification.
- Model: Headless EfficientNetBO (train only top layer, freeze everything else)
- Data: 750 training images, 625 test images (validation step is 2500×0.25).
- number of classes. 10 (prepared from the Food101 dataset).
- Number of epochs: 5
- Batch size: 4 (Lower batch size was required due to M1 not having memory capacity to handle higher size. Tried batch size 32, 16, 8, all failed).
3.3 tensorflow_macos GitHub benchmark
While browsing Apple’s tensorflow _macos GitHub, I came across an Issues thread with quite a few people running benchmarks on various machines. So I decided to incorporate it into my testing.
- Challenge: Multi-class image classification
- Model: LeNet
- 60,000 data training images, 10,000 test images (provided by MNIST).
- Number of classes: 10
- Number of epochs: 12
- Batch size: 128
- Source: https://github.com/apple/tensorflow_macos/issues/25
TensorFlow code result
I not only ran the above three experiments on every MacBook, but also on a Google Colab instance with a GPU running solid (my usual workflow is to run the experiments on Google Colab and use the scale up with larger cloud servers depending on demand).
| M1 chip MacBook Air | M1 chip 13-inch MacBook Pro | Intel-based 16-inch MacBook Pro | T4 GPU chip Google Colab | |
| Basic CNN | 7-8 seconds/epoch | 7-8 seconds/epoch | 35-41 sec/epoch | 5 seconds/epoch |
| transfer learning | 20-21 sec/epoch | 20-21 sec/epoch | 59-66 sec/epoch | 7 seconds/epoch |
| tensorflow _macos benchmark | 23-24 seconds/epoch | 25-26 seconds/epoch | 20-21 sec/epoch | 9 seconds/epoch |
(Table: Training results when running TensorFlow code) Google Colab was the fastest, but the M1 MacBook wasn’t too slow either. The M1 MacBook can be very useful for running tests and small experiments locally without having to be constantly connected to Colab. Note: For any model, the first epoch is often the longest due to data loading, so all but the first epoch are included in the velocity measurements.
Google Colab’s GPU instance used pure TensorFlow instead of tensorflow _macos .
The instance running on Google Colab GPU had the fastest performance in all three tests.
Notably, the M1 machine significantly outperformed the Intel machine in basic CNN and transfer learning experiments.
However, on Intel-powered machines, the tensorflow _macos benchmark showed some recovery. This result seems to be due to explicitly telling TensorFlow to use the GPU by using the tensorflow _macos code.
I also ran some basic CNN and transfer learning experiments with the code explicitly telling it to use the GPU and it made no difference. These results are probably related to the different data loading schemes used in the two experimental setups and the tensorflow _macos benchmark.
See the attached Google Colab Notebook for all the code used in the experiment .
portability
If you bought a laptop, you want to carry it around. Maybe you’re writing while you’re looking at the beach, or you’re sipping coffee at your local café and coding your latest experiments.
So, to compare the portability of the three Macs, here’s the battery usage for each of the three tests, and the new portability score I devised.
| M1 chip MacBook Air | M1 chip 13-inch MacBook Pro | Intel-based 16-inch MacBook Pro | |
| Battery level at start | 89% | 92% | 87% |
| End battery level | 39% | 35% | 65%* |
| Battery level change | -50% | -57% | -122% |
| Portability Score** | 64.5 | 79.8 | 244 |
(Table: Battery consumption and portability score) All experiments were conducted over two days (about 3 hours on the first day and about 2 hours on the second day). And the M1 machine never charged.
- *16-inch MacBook Pro fully charged (from 0%) once.
- **Portability score = battery life/weight ratio = battery life lost x weight (lower is better).
All things considered, M1-chip Macs crushed Intel-chip Macs. Two models of Macs with the M1 chip finished all tests with more than 30% battery life. The MacBook Air stands out again, drawing the least amount of power and scoring the lowest points for portability.
16-inch MacBook Pro sold off
Anyone want to buy a second-hand MacBook Pro 16-inch that is close to top specs? Well taken care of, I promise.
After doing the above experiment and using the M1 chip-equipped MacBook for a few weeks, the graphs Apple provided seemed pretty realistic. Also, are all the reviews praising these MacBooks true? Well, in my experience, these reviews are correct.
I originally purchased the 16-inch MacBook Pro as a high-spec laptop for home use, or a high-speed machine. And it did. However, the M1 machine is also high spec and high speed. Plus, the M1 machine is light, quiet, and has good battery life.
The only test the MacBook Pro 16-inch won was the video rendering test (and screen size, of course). Still, the results weren’t noticeably better, and it didn’t convince me that “this car costs 2.5 times more”.
What next?
I have a 13 inch MacBook Pro. This Mac beats the 13-inch M1 chip Air with a few extras for video editing (plus, I edited all the video versions of this article on the 13-inch Pro, and never had a problem. rice field). I will use this as a portable machine and will probably set up the 16″ as a permanent desktop PC.
But the Air… oh, the Air… I remember when I was working at an Apple store, telling visitors to buy an Air if they were just word processing and web browsing. Now we can train machine learning models.
So which one should you get?
“I write code, write text, browse the web” or “I just want a portable and capable machine”
If that’s all you want, get the M1 chip MacBook Air. You won’t be disappointed. If video editing wasn’t my daily routine, I would have gotten this too.
“Write code, write sentences, browse the web, edit videos, etc.”
In my testing, the 13-inch M1 MacBook Pro and M1 MacBook Air perform 70-90% of the nearly top-spec Intel 16-inch MacBook Pro, so both are phenomenal performance. If you want to put some effort into video editing, the 13-inch M1 MacBook Pro might be a good choice.
“I want a bigger screen size, I don’t care about the cost”
For now, the only valid reason to consider buying an Intel-based 16-inch MacBook Pro would be if screen size is paramount and you don’t have the budget for an external monitor.
Screen size isn’t that important to me. Most of the time I’m either running one app in full screen or running two apps split in the middle of the screen. Connect your machine to an external monitor if you want to run multiple applications (Note: Currently, M1 Macs only support one external monitor, so 3 or more monitor fans require an Intel-based Mac will be).
It’s already been said that if you want an M1 chip MacBook larger than 13 inches, hold off buying now and wait until the 16-inch Apple silicon MacBook is released.
Conclusion
As I mentioned earlier, my 16-inch MacBook Pro had no issues. But compared to the new M1 MacBook, it feels dated.
I haven’t been this impressed with a new computer since switching from a hard disk to a solid state drive for the first time.
As I write this, the 13-inch M1 MacBook Pro feels buttery smooth. The little things, the good things, the new keyboard style, the native Apple apps, the battery life, all add up to a pleasing experience.
If Apple can achieve such performance gains in laptops with first-generation M1 chips (even fanless ones), it’s hard to imagine what it would be like in power-unconstrained machines (Macs with M1 chips mini hints at that possibility).
