Tutorial¶
To begin running a neural network training, first make sure that we have our iterable of batches. We are free to use anything as a batch. For instance, if PyTorch is used, we may represent the iterable of batches as a DataLoader object.
Once the batches are ready, we create a Runner
object.
from rnnr import Runner
trainer = Runner()
Next, we give the runner a function that is invoked on each batch of the run. For example, if PyTorch is used, we can do it like below:
from rnnr import Event
import torch.nn.functional as F
@trainer.on(Event.BATCH)
def train_update(state):
x, t = state['batch']
y = model(x)
loss = F.cross_entropy(y, t)
optimizer.zero_grad()
loss.backward()
optimizer.step()
To run the trainer, simply invoke the run
method.
trainer.run(batches, max_epoch=5)
Listening to other events¶
The above example is the minimum requirement to use a Runner
. However, usually we want to
have some more control of what we do on the start of each epoch, end of each batch, and so on.
Heavily inspired by torchnet and Ignite, rnnr also uses an event system: during a run,
the runner emits events and we can provide callbacks for them.
@trainer.on(Event.STARTED)
def print_training(state):
print('Start training')
@trainer.on(Event.EPOCH_STARTED)
def print_epoch(state):
print('Epoch', state['epoch'], 'started')
Now, when trainer
is run, these callbacks will be invoked at the start of the run and the
start of each epoch respectively. The on
method can also be used not as a decorator
by passing a callback as its second argument.
def print_epoch_finished(state):
print('Epoch', state['epoch'], 'of', state['max_epoch'], 'finished')
trainer.on(Event.EPOCH_FINISHED, print_epoch_finished)
See Runner
for more details. For more information on what events are available, see Event
instead. Please check Callbacks for some useful callback factories.
Callbacks that work together: an attachment¶
An attachment is a collection of callbacks that work together to provide some functionality.
For example, to compute a mean over batch statistics, we need to append callbacks to many
events, and these callbacks work together to obtain the mean value. In rnnr, this concept
is realized by the Attachment
abstract base class. We can create our own
attachments, but some useful attachments are provided. See Attachments for more.