Listen Attend Spell Model

Listen Attend Spell Model

class openspeech.models.listen_attend_spell.model.DeepCNNWithJointCTCListenAttendSpellModel(configs: omegaconf.dictconfig.DictConfig, tokenizer: openspeech.tokenizers.tokenizer.Tokenizer)[source]

Listen, Attend and Spell model with configurable encoder and decoder. Paper: https://arxiv.org/abs/1508.01211

Parameters

  • configs (DictConfig) – configuration set.

  • tokenizer (Tokenizer) – tokenizer is in charge of preparing the inputs for a model.

Inputs:

  • inputs (torch.FloatTensor): A input sequence passed to encoders. Typically for inputs this will be a padded FloatTensor of size (batch, seq_length, dimension).

  • input_lengths (torch.LongTensor): The length of input tensor. (batch)

Returns

Result of model predictions.

Return type

outputs (dict)

set_beam_decoder(beam_size: int = 3)[source]

Setting beam search decoder

class openspeech.models.listen_attend_spell.model.JointCTCListenAttendSpellModel(configs: omegaconf.dictconfig.DictConfig, tokenizer: openspeech.tokenizers.tokenizer.Tokenizer)[source]

Joint CTC-Attention Listen, Attend and Spell model with configurable encoder and decoder. Paper: https://arxiv.org/abs/1609.06773

Parameters

  • configs (DictConfig) – configuration set.

  • tokenizer (Tokeizer) – tokenizer is in charge of preparing the inputs for a model.

Inputs:

  • inputs (torch.FloatTensor): A input sequence passed to encoders. Typically for inputs this will be a padded FloatTensor of size (batch, seq_length, dimension).

  • input_lengths (torch.LongTensor): The length of input tensor. (batch)

Returns

Result of model predictions.

Return type

outputs (dict)

set_beam_decoder(beam_size: int = 3)[source]

Setting beam search decoder

class openspeech.models.listen_attend_spell.model.ListenAttendSpellModel(configs: omegaconf.dictconfig.DictConfig, tokenizer: openspeech.tokenizers.tokenizer.Tokenizer)[source]

Listen, Attend and Spell model with configurable encoder and decoder. Paper: https://arxiv.org/abs/1508.01211

Parameters

  • configs (DictConfig) – configuration set.

  • tokenizer (Tokeizer) – tokenizer is in charge of preparing the inputs for a model.

Inputs:

  • inputs (torch.FloatTensor): A input sequence passed to encoders. Typically for inputs this will be a padded FloatTensor of size (batch, seq_length, dimension).

  • input_lengths (torch.LongTensor): The length of input tensor. (batch)

Returns

Result of model predictions.

Return type

outputs (dict)

set_beam_decoder(beam_size: int = 3)[source]

Setting beam search decoder

class openspeech.models.listen_attend_spell.model.ListenAttendSpellWithLocationAwareModel(configs: omegaconf.dictconfig.DictConfig, tokenizer: openspeech.tokenizers.tokenizer.Tokenizer)[source]

Listen, Attend and Spell model with configurable encoder and decoder. Paper: https://arxiv.org/abs/1508.01211

Parameters

  • configs (DictConfig) – configuration set.

  • tokenizer (Tokeizer) – tokenizer is in charge of preparing the inputs for a model.

Inputs:

  • inputs (torch.FloatTensor): A input sequence passed to encoders. Typically for inputs this will be a padded FloatTensor of size (batch, seq_length, dimension).

  • input_lengths (torch.LongTensor): The length of input tensor. (batch)

Returns

Result of model predictions.

Return type

outputs (dict)

set_beam_decoder(beam_size: int = 3)[source]

Setting beam search decoder

class openspeech.models.listen_attend_spell.model.ListenAttendSpellWithMultiHeadModel(configs: omegaconf.dictconfig.DictConfig, tokenizer: openspeech.tokenizers.tokenizer.Tokenizer)[source]

Listen, Attend and Spell model with configurable encoder and decoder. Paper: https://arxiv.org/abs/1508.01211

Parameters

  • configs (DictConfig) – configuration set.

  • tokenizer (Tokeizer) – tokenizer is in charge of preparing the inputs for a model.

Inputs:

  • inputs (torch.FloatTensor): A input sequence passed to encoders. Typically for inputs this will be a padded FloatTensor of size (batch, seq_length, dimension).

  • input_lengths (torch.LongTensor): The length of input tensor. (batch)

Returns

Result of model predictions.

Return type

outputs (dict)

set_beam_decoder(beam_size: int = 3)[source]

Setting beam search decoder

Listen Attend Spell Model Configuration

class openspeech.models.listen_attend_spell.configurations.DeepCNNWithJointCTCListenAttendSpellConfigs(model_name: str = 'deep_cnn_with_joint_ctc_listen_attend_spell', num_encoder_layers: int = 3, num_decoder_layers: int = 2, hidden_state_dim: int = 768, encoder_dropout_p: float = 0.3, encoder_bidirectional: bool = True, rnn_type: str = 'lstm', extractor: str = 'vgg', activation: str = 'hardtanh', joint_ctc_attention: bool = True, max_length: int = 128, num_attention_heads: int = 1, decoder_dropout_p: float = 0.2, decoder_attn_mechanism: str = 'loc', teacher_forcing_ratio: float = 1.0, optimizer: str = 'adam')[source]

This is the configuration class to store the configuration of a DeepCNNWithJointCTCListenAttendSpell.

It is used to initiated an DeepCNNWithJointCTCListenAttendSpell model.

Configuration objects inherit from :class: ~openspeech.dataclass.configs.OpenspeechDataclass.

Parameters

  • model_name (str) – Model name (default: deep_cnn_with_joint_ctc_listen_attend_spell)

  • num_encoder_layers (int) – The number of encoder layers. (default: 3)

  • num_decoder_layers (int) – The number of decoder layers. (default: 2)

  • hidden_state_dim (int) – The hidden state dimension of encoder. (default: 768)

  • encoder_dropout_p (float) – The dropout probability of encoder. (default: 0.3)

  • encoder_bidirectional (bool) – If True, becomes a bidirectional encoders (default: True)

  • rnn_type (str) – Type of rnn cell (rnn, lstm, gru) (default: lstm)

  • extractor (str) – The CNN feature extractor. (default: vgg)

  • activation (str) – Type of activation function (default: str)

  • joint_ctc_attention (bool) – Flag indication joint ctc attention or not (default: True)

  • max_length (int) – Max decoding length. (default: 128)

  • num_attention_heads (int) – The number of attention heads. (default: 1)

  • decoder_dropout_p (float) – The dropout probability of decoder. (default: 0.2)

  • decoder_attn_mechanism (str) – The attention mechanism for decoder. (default: loc)

  • teacher_forcing_ratio (float) – The ratio of teacher forcing. (default: 1.0)

  • optimizer (str) – Optimizer for training. (default: adam)

class openspeech.models.listen_attend_spell.configurations.JointCTCListenAttendSpellConfigs(model_name: str = 'joint_ctc_listen_attend_spell', num_encoder_layers: int = 3, num_decoder_layers: int = 2, hidden_state_dim: int = 768, encoder_dropout_p: float = 0.3, encoder_bidirectional: bool = True, rnn_type: str = 'lstm', joint_ctc_attention: bool = True, max_length: int = 128, num_attention_heads: int = 1, decoder_dropout_p: float = 0.2, decoder_attn_mechanism: str = 'loc', teacher_forcing_ratio: float = 1.0, optimizer: str = 'adam')[source]

This is the configuration class to store the configuration of a JointCTCListenAttendSpell.

It is used to initiated an JointCTCListenAttendSpell model.

Configuration objects inherit from :class: ~openspeech.dataclass.configs.OpenspeechDataclass.

Parameters

  • model_name (str) – Model name (default: joint_ctc_listen_attend_spell)

  • num_encoder_layers (int) – The number of encoder layers. (default: 3)

  • num_decoder_layers (int) – The number of decoder layers. (default: 2)

  • hidden_state_dim (int) – The hidden state dimension of encoder. (default: 768)

  • encoder_dropout_p (float) – The dropout probability of encoder. (default: 0.3)

  • encoder_bidirectional (bool) – If True, becomes a bidirectional encoders (default: True)

  • rnn_type (str) – Type of rnn cell (rnn, lstm, gru) (default: lstm)

  • joint_ctc_attention (bool) – Flag indication joint ctc attention or not (default: True)

  • max_length (int) – Max decoding length. (default: 128)

  • num_attention_heads (int) – The number of attention heads. (default: 1)

  • decoder_dropout_p (float) – The dropout probability of decoder. (default: 0.2)

  • decoder_attn_mechanism (str) – The attention mechanism for decoder. (default: loc)

  • teacher_forcing_ratio (float) – The ratio of teacher forcing. (default: 1.0)

  • optimizer (str) – Optimizer for training. (default: adam)

class openspeech.models.listen_attend_spell.configurations.ListenAttendSpellConfigs(model_name: str = 'listen_attend_spell', num_encoder_layers: int = 3, num_decoder_layers: int = 2, hidden_state_dim: int = 512, encoder_dropout_p: float = 0.3, encoder_bidirectional: bool = True, rnn_type: str = 'lstm', joint_ctc_attention: bool = False, max_length: int = 128, num_attention_heads: int = 1, decoder_dropout_p: float = 0.2, decoder_attn_mechanism: str = 'dot', teacher_forcing_ratio: float = 1.0, optimizer: str = 'adam')[source]

This is the configuration class to store the configuration of a ListenAttendSpell.

It is used to initiated an ListenAttendSpell model.

Configuration objects inherit from :class: ~openspeech.dataclass.configs.OpenspeechDataclass.

Parameters

  • model_name (str) – Model name (default: listen_attend_spell)

  • num_encoder_layers (int) – The number of encoder layers. (default: 3)

  • num_decoder_layers (int) – The number of decoder layers. (default: 2)

  • hidden_state_dim (int) – The hidden state dimension of encoder. (default: 512)

  • encoder_dropout_p (float) – The dropout probability of encoder. (default: 0.3)

  • encoder_bidirectional (bool) – If True, becomes a bidirectional encoders (default: True)

  • rnn_type (str) – Type of rnn cell (rnn, lstm, gru) (default: lstm)

  • joint_ctc_attention (bool) – Flag indication joint ctc attention or not (default: False)

  • max_length (int) – Max decoding length. (default: 128)

  • num_attention_heads (int) – The number of attention heads. (default: 1)

  • decoder_dropout_p (float) – The dropout probability of decoder. (default: 0.2)

  • decoder_attn_mechanism (str) – The attention mechanism for decoder. (default: dot)

  • teacher_forcing_ratio (float) – The ratio of teacher forcing. (default: 1.0)

  • optimizer (str) – Optimizer for training. (default: adam)

class openspeech.models.listen_attend_spell.configurations.ListenAttendSpellWithLocationAwareConfigs(model_name: str = 'listen_attend_spell_with_location_aware', num_encoder_layers: int = 3, num_decoder_layers: int = 2, hidden_state_dim: int = 512, encoder_dropout_p: float = 0.3, encoder_bidirectional: bool = True, rnn_type: str = 'lstm', joint_ctc_attention: bool = False, max_length: int = 128, num_attention_heads: int = 1, decoder_dropout_p: float = 0.2, decoder_attn_mechanism: str = 'loc', teacher_forcing_ratio: float = 1.0, optimizer: str = 'adam')[source]

This is the configuration class to store the configuration of a ListenAttendSpellWithLocationAware.

It is used to initiated an ListenAttendSpellWithLocationAware model.

Configuration objects inherit from :class: ~openspeech.dataclass.configs.OpenspeechDataclass.

Parameters

  • model_name (str) – Model name (default: listen_attend_spell_with_location_aware)

  • num_encoder_layers (int) – The number of encoder layers. (default: 3)

  • num_decoder_layers (int) – The number of decoder layers. (default: 2)

  • hidden_state_dim (int) – The hidden state dimension of encoder. (default: 512)

  • encoder_dropout_p (float) – The dropout probability of encoder. (default: 0.3)

  • encoder_bidirectional (bool) – If True, becomes a bidirectional encoders (default: True)

  • rnn_type (str) – Type of rnn cell (rnn, lstm, gru) (default: lstm)

  • joint_ctc_attention (bool) – Flag indication joint ctc attention or not (default: False)

  • max_length (int) – Max decoding length. (default: 128)

  • num_attention_heads (int) – The number of attention heads. (default: 1)

  • decoder_dropout_p (float) – The dropout probability of decoder. (default: 0.2)

  • decoder_attn_mechanism (str) – The attention mechanism for decoder. (default: loc)

  • teacher_forcing_ratio (float) – The ratio of teacher forcing. (default: 1.0)

  • optimizer (str) – Optimizer for training. (default: adam)

class openspeech.models.listen_attend_spell.configurations.ListenAttendSpellWithMultiHeadConfigs(model_name: str = 'listen_attend_spell_with_multi_head', num_encoder_layers: int = 3, num_decoder_layers: int = 2, hidden_state_dim: int = 512, encoder_dropout_p: float = 0.3, encoder_bidirectional: bool = True, rnn_type: str = 'lstm', joint_ctc_attention: bool = False, max_length: int = 128, num_attention_heads: int = 4, decoder_dropout_p: float = 0.2, decoder_attn_mechanism: str = 'multi-head', teacher_forcing_ratio: float = 1.0, optimizer: str = 'adam')[source]

This is the configuration class to store the configuration of a ListenAttendSpellWithMultiHead.

It is used to initiated an ListenAttendSpellWithMultiHead model.

Configuration objects inherit from :class: ~openspeech.dataclass.configs.OpenspeechDataclass.

Parameters

  • model_name (str) – Model name (default: listen_attend_spell_with_multi_head)

  • num_encoder_layers (int) – The number of encoder layers. (default: 3)

  • num_decoder_layers (int) – The number of decoder layers. (default: 2)

  • hidden_state_dim (int) – The hidden state dimension of encoder. (default: 512)

  • encoder_dropout_p (float) – The dropout probability of encoder. (default: 0.3)

  • encoder_bidirectional (bool) – If True, becomes a bidirectional encoders (default: True)

  • rnn_type (str) – Type of rnn cell (rnn, lstm, gru) (default: lstm)

  • joint_ctc_attention (bool) – Flag indication joint ctc attention or not (default: False)

  • max_length (int) – Max decoding length. (default: 128)

  • num_attention_heads (int) – The number of attention heads. (default: 4)

  • decoder_dropout_p (float) – The dropout probability of decoder. (default: 0.2)

  • decoder_attn_mechanism (str) – The attention mechanism for decoder. (default: multi-head)

  • teacher_forcing_ratio (float) – The ratio of teacher forcing. (default: 1.0)

  • optimizer (str) – Optimizer for training. (default: adam)