For training and deploying complex models in Artificial Intelligence, which one among the following is preferred ?

Correct — B, (b) Graphics Processing Units. Training a large model is arithmetic on a scale that suits a processor built for doing the same operation on thousands of numbers at once, and that is what a graphics processing unit is.

Why the graphics chip won this work. Almost everything inside a neural network is matrix multiplication: multiply a block of numbers by another block, add a bias, apply a simple function, repeat for every layer and for every example in the batch. Those multiplications do not depend on one another, so they can all be done at the same time. A central processing unit has a few very capable cores designed to run one complicated instruction stream quickly, with branching and prediction; a graphics processing unit has thousands of simple cores designed to apply one operation to a great many values simultaneously, which is what drawing an image also demands. Add very high memory bandwidth to keep those cores fed, and the same hardware that renders a scene turns out to be the ideal engine for training a model. Modern accelerators go further and include units specialised for the matrix operations themselves.

That is why the training of large models is measured in GPU-hours, why clusters of these accelerators are the constraint on who can build such models, and why the same hardware is used again for deployment — running a trained model for many users at once is also a parallel job, even if a lighter one than training.

Read the two halves of the stem together: it says 'training and deploying complex models'. Both phases are the same kind of arithmetic, and both are done on the same class of processor.

Why the others are wrong

• (a)Central Processing Units — The general-purpose processor of a computer, and the one that runs the operating system and coordinates everything, including the accelerators. It is not preferred for this work because its strength is the wrong strength: a handful of fast, flexible cores handling complicated sequential logic. For the enormous number of identical, independent multiplications that training requires, it is far slower than a processor with thousands of simple cores. It remains essential — but as the manager of the job, not as the engine.
• (c)Read Only Memory — Not a processor at all. Read-only memory is a form of non-volatile storage holding firmware such as the start-up instructions of a computer; it does no arithmetic and it is not written to in ordinary operation. Its presence in this option set is a category error offered to a candidate matching abbreviations rather than reading them.
• (d)Solid State Drives — Storage, not computation. Fast storage genuinely matters when training a model, because enormous datasets have to be streamed to the processors and checkpoints written out, so this option touches something real. But the drive holds the data; the arithmetic is done elsewhere. The question asks what is preferred for training and deploying, and the answer to that has to be a processor.

Computers separate the work of processing from the work of storing. The central processing unit executes instructions and is optimised for latency — getting one complicated task done as fast as possible. A graphics processing unit is optimised for throughput — getting a very large number of simple, identical tasks done at once — with thousands of small cores and wide memory bandwidth. Memory is arranged in a hierarchy: registers and cache inside the processor, then random access memory, which is volatile and loses its contents when power is removed, then non-volatile storage such as solid state drives and hard disks, and read-only memory holding firmware. Machine learning workloads sit almost entirely in the throughput category, which is why accelerators rather than general-purpose processors define the field's economics.

Key facts

• A graphics processing unit has thousands of simple cores and high memory bandwidth, suited to performing the same operation on many values in parallel.
• Neural network training and inference consist largely of matrix multiplications, which parallelise well.
• A central processing unit has a few powerful cores optimised for sequential, branching work and coordinates the system.
• Read-only memory is non-volatile storage for firmware and performs no computation.
• Solid state drives are secondary storage; they hold data and models but do not process them.
• Random access memory is volatile primary memory, faster than secondary storage and directly accessible to the processor.
• Training cost for large models is commonly measured in processor-hours on such accelerators.
• Specialised accelerators designed purely for tensor operations exist alongside graphics processing units and are used for the same workloads.

Common traps

• Choosing the central processing unit because it is the best-known processor; the question is about the kind of work, not about prominence.
• Confusing memory and storage with processing components.
• Assuming the graphics processor is only for displays; graphics is where it began, not where it is confined.
• Overlooking that the stem asks about both training and deployment, which point to the same answer.

• practice — not a real PYQ

  Which one among the following best explains why graphics processing units are preferred for training neural networks ?

  • (a)They store more data than solid state drives
  • (b)They perform a very large number of identical arithmetic operations in parallel
  • (c)They retain their contents when power is switched off
  • (d)They translate high level languages into machine code

  Answer(b) They perform a very large number of identical arithmetic operations in parallel