Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.8/site-packages/tensorflow/_api/v2/nn/__init__.py: 100%

1# This file is MACHINE GENERATED! Do not edit. 

2# Generated by: tensorflow/python/tools/api/generator/create_python_api.py script. 

3"""Primitive Neural Net (NN) Operations. 

4 

5## Notes on padding 

6 

7Several neural network operations, such as `tf.nn.conv2d` and 

8`tf.nn.max_pool2d`, take a `padding` parameter, which controls how the input is 

9padded before running the operation. The input is padded by inserting values 

10(typically zeros) before and after the tensor in each spatial dimension. The 

11`padding` parameter can either be the string `'VALID'`, which means use no 

12padding, or `'SAME'` which adds padding according to a formula which is 

13described below. Certain ops also allow the amount of padding per dimension to 

14be explicitly specified by passing a list to `padding`. 

15 

16In the case of convolutions, the input is padded with zeros. In case of pools, 

17the padded input values are ignored. For example, in a max pool, the sliding 

18window ignores padded values, which is equivalent to the padded values being 

19`-infinity`. 

20 

21### `'VALID'` padding 

22 

23Passing `padding='VALID'` to an op causes no padding to be used. This causes the 

24output size to typically be smaller than the input size, even when the stride is 

25one. In the 2D case, the output size is computed as: 

26 

27```python 

28out_height = ceil((in_height - filter_height + 1) / stride_height) 

29out_width = ceil((in_width - filter_width + 1) / stride_width) 

30``` 

31 

32The 1D and 3D cases are similar. Note `filter_height` and `filter_width` refer 

33to the filter size after dilations (if any) for convolutions, and refer to the 

34window size for pools. 

35 

36### `'SAME'` padding 

37 

38With `'SAME'` padding, padding is applied to each spatial dimension. When the 

39strides are 1, the input is padded such that the output size is the same as the 

40input size. In the 2D case, the output size is computed as: 

41 

42```python 

43out_height = ceil(in_height / stride_height) 

44out_width = ceil(in_width / stride_width) 

45``` 

46 

47The amount of padding used is the smallest amount that results in the output 

48size. The formula for the total amount of padding per dimension is: 

49 

50```python 

51if (in_height % strides[1] == 0): 

52 pad_along_height = max(filter_height - stride_height, 0) 

53else: 

54 pad_along_height = max(filter_height - (in_height % stride_height), 0) 

55if (in_width % strides[2] == 0): 

56 pad_along_width = max(filter_width - stride_width, 0) 

57else: 

58 pad_along_width = max(filter_width - (in_width % stride_width), 0) 

59``` 

60 

61Finally, the padding on the top, bottom, left and right are: 

62 

63```python 

64pad_top = pad_along_height // 2 

65pad_bottom = pad_along_height - pad_top 

66pad_left = pad_along_width // 2 

67pad_right = pad_along_width - pad_left 

68``` 

69 

70Note that the division by 2 means that there might be cases when the padding on 

71both sides (top vs bottom, right vs left) are off by one. In this case, the 

72bottom and right sides always get the one additional padded pixel. For example, 

73when pad_along_height is 5, we pad 2 pixels at the top and 3 pixels at the 

74bottom. Note that this is different from existing libraries such as PyTorch and 

75Caffe, which explicitly specify the number of padded pixels and always pad the 

76same number of pixels on both sides. 

77 

78Here is an example of `'SAME'` padding: 

79 

80>>> in_height = 5 

81>>> filter_height = 3 

82>>> stride_height = 2 

83>>> 

84>>> in_width = 2 

85>>> filter_width = 2 

86>>> stride_width = 1 

87>>> 

88>>> inp = tf.ones((2, in_height, in_width, 2)) 

89>>> filter = tf.ones((filter_height, filter_width, 2, 2)) 

90>>> strides = [stride_height, stride_width] 

91>>> output = tf.nn.conv2d(inp, filter, strides, padding='SAME') 

92>>> output.shape[1] # output_height: ceil(5 / 2) 

933 

94>>> output.shape[2] # output_width: ceil(2 / 1) 

952 

96 

97### Explicit padding 

98 

99Certain ops, like `tf.nn.conv2d`, also allow a list of explicit padding amounts 

100to be passed to the `padding` parameter. This list is in the same format as what 

101is passed to `tf.pad`, except the padding must be a nested list, not a tensor. 

102For example, in the 2D case, the list is in the format `[[0, 0], [pad_top, 

103pad_bottom], [pad_left, pad_right], [0, 0]]` when `data_format` is its default 

104value of `'NHWC'`. The two `[0, 0]` pairs indicate the batch and channel 

105dimensions have no padding, which is required, as only spatial dimensions can 

106have padding. 

107 

108For example: 

109 

110>>> inp = tf.ones((1, 3, 3, 1)) 

111>>> filter = tf.ones((2, 2, 1, 1)) 

112>>> strides = [1, 1] 

113>>> padding = [[0, 0], [1, 2], [0, 1], [0, 0]] 

114>>> output = tf.nn.conv2d(inp, filter, strides, padding=padding) 

115>>> tuple(output.shape) 

116(1, 5, 3, 1) 

117>>> # Equivalently, tf.pad can be used, since convolutions pad with zeros. 

118>>> inp = tf.pad(inp, padding) 

119>>> # 'VALID' means to use no padding in conv2d (we already padded inp) 

120>>> output2 = tf.nn.conv2d(inp, filter, strides, padding='VALID') 

121>>> tf.debugging.assert_equal(output, output2) 

122 

123### Difference between convolution and pooling layers 

124How padding is used in convolution layers and pooling layers is different. For 

125convolution layers, padding is filled with values of zero, and padding is 

126multiplied with kernels. For pooling layers, padding is excluded from the 

127computation. For example when applying average pooling to a 4x4 grid, how much 

128padding is added will not impact the output. Here is an example that 

129demonstrates the difference. 

130 

131>>> x_in = np.array([[ 

132... [[2], [2]], 

133... [[1], [1]], 

134... [[1], [1]]]]) 

135>>> kernel_in = np.array([ # simulate the avg_pool with conv2d 

136... [ [[0.25]], [[0.25]] ], 

137... [ [[0.25]], [[0.25]] ]]) 

138>>> x = tf.constant(x_in, dtype=tf.float32) 

139>>> kernel = tf.constant(kernel_in, dtype=tf.float32) 

140>>> conv_out = tf.nn.conv2d(x, kernel, strides=[1, 1, 1, 1], padding='SAME') 

141>>> pool_out = tf.nn.avg_pool(x, [2, 2], strides=[1, 1, 1, 1], padding='SAME') 

142>>> print(conv_out.shape, pool_out.shape) 

143(1, 3, 2, 1) (1, 3, 2, 1) 

144>>> tf.reshape(conv_out, [3, 2]).numpy() # conv2d takes account of padding 

145array([[1.5 , 0.75], 

146 [1. , 0.5 ], 

147 [0.5 , 0.25]], dtype=float32) 

148>>> tf.reshape(pool_out, [3, 2]).numpy() # avg_pool excludes padding 

149array([[1.5, 1.5], 

150 [1. , 1. ], 

151 [1. , 1. ]], dtype=float32) 

152 

153 

154""" 

155 

156import sys as _sys 

157 

158from . import experimental 

159from tensorflow.python.keras.layers.rnn_cell_wrapper_v2 import DeviceWrapper as RNNCellDeviceWrapper 

160from tensorflow.python.keras.layers.rnn_cell_wrapper_v2 import DropoutWrapper as RNNCellDropoutWrapper 

161from tensorflow.python.keras.layers.rnn_cell_wrapper_v2 import ResidualWrapper as RNNCellResidualWrapper 

162from tensorflow.python.ops.array_ops import depth_to_space_v2 as depth_to_space 

163from tensorflow.python.ops.array_ops import space_to_batch_v2 as space_to_batch 

164from tensorflow.python.ops.array_ops import space_to_depth_v2 as space_to_depth 

165from tensorflow.python.ops.candidate_sampling_ops import all_candidate_sampler 

166from tensorflow.python.ops.candidate_sampling_ops import compute_accidental_hits 

167from tensorflow.python.ops.candidate_sampling_ops import fixed_unigram_candidate_sampler 

168from tensorflow.python.ops.candidate_sampling_ops import learned_unigram_candidate_sampler 

169from tensorflow.python.ops.ctc_ops import collapse_repeated 

170from tensorflow.python.ops.ctc_ops import ctc_beam_search_decoder_v2 as ctc_beam_search_decoder 

171from tensorflow.python.ops.ctc_ops import ctc_greedy_decoder 

172from tensorflow.python.ops.ctc_ops import ctc_loss_v3 as ctc_loss 

173from tensorflow.python.ops.ctc_ops import ctc_unique_labels 

174from tensorflow.python.ops.embedding_ops import embedding_lookup_sparse_v2 as embedding_lookup_sparse 

175from tensorflow.python.ops.embedding_ops import embedding_lookup_v2 as embedding_lookup 

176from tensorflow.python.ops.embedding_ops import safe_embedding_lookup_sparse_v2 as safe_embedding_lookup_sparse 

177from tensorflow.python.ops.gen_math_ops import tanh 

178from tensorflow.python.ops.gen_nn_ops import elu 

179from tensorflow.python.ops.gen_nn_ops import l2_loss 

180from tensorflow.python.ops.gen_nn_ops import lrn 

181from tensorflow.python.ops.gen_nn_ops import lrn as local_response_normalization 

182from tensorflow.python.ops.gen_nn_ops import relu 

183from tensorflow.python.ops.gen_nn_ops import selu 

184from tensorflow.python.ops.gen_nn_ops import softsign 

185from tensorflow.python.ops.math_ops import sigmoid 

186from tensorflow.python.ops.math_ops import softplus 

187from tensorflow.python.ops.nn_impl import batch_norm_with_global_normalization_v2 as batch_norm_with_global_normalization 

188from tensorflow.python.ops.nn_impl import batch_normalization 

189from tensorflow.python.ops.nn_impl import compute_average_loss 

190from tensorflow.python.ops.nn_impl import depthwise_conv2d_v2 as depthwise_conv2d 

191from tensorflow.python.ops.nn_impl import l2_normalize 

192from tensorflow.python.ops.nn_impl import log_poisson_loss 

193from tensorflow.python.ops.nn_impl import moments_v2 as moments 

194from tensorflow.python.ops.nn_impl import nce_loss_v2 as nce_loss 

195from tensorflow.python.ops.nn_impl import normalize_moments 

196from tensorflow.python.ops.nn_impl import sampled_softmax_loss_v2 as sampled_softmax_loss 

197from tensorflow.python.ops.nn_impl import scale_regularization_loss 

198from tensorflow.python.ops.nn_impl import separable_conv2d_v2 as separable_conv2d 

199from tensorflow.python.ops.nn_impl import sigmoid_cross_entropy_with_logits_v2 as sigmoid_cross_entropy_with_logits 

200from tensorflow.python.ops.nn_impl import sufficient_statistics_v2 as sufficient_statistics 

201from tensorflow.python.ops.nn_impl import swish 

202from tensorflow.python.ops.nn_impl import swish as silu 

203from tensorflow.python.ops.nn_impl import weighted_cross_entropy_with_logits_v2 as weighted_cross_entropy_with_logits 

204from tensorflow.python.ops.nn_impl import weighted_moments_v2 as weighted_moments 

205from tensorflow.python.ops.nn_impl import zero_fraction 

206from tensorflow.python.ops.nn_ops import approx_max_k 

207from tensorflow.python.ops.nn_ops import approx_min_k 

208from tensorflow.python.ops.nn_ops import atrous_conv2d 

209from tensorflow.python.ops.nn_ops import atrous_conv2d_transpose 

210from tensorflow.python.ops.nn_ops import avg_pool1d 

211from tensorflow.python.ops.nn_ops import avg_pool2d 

212from tensorflow.python.ops.nn_ops import avg_pool3d 

213from tensorflow.python.ops.nn_ops import avg_pool_v2 as avg_pool 

214from tensorflow.python.ops.nn_ops import bias_add 

215from tensorflow.python.ops.nn_ops import conv1d_transpose 

216from tensorflow.python.ops.nn_ops import conv1d_v2 as conv1d 

217from tensorflow.python.ops.nn_ops import conv2d_transpose_v2 as conv2d_transpose 

218from tensorflow.python.ops.nn_ops import conv2d_v2 as conv2d 

219from tensorflow.python.ops.nn_ops import conv3d_transpose_v2 as conv3d_transpose 

220from tensorflow.python.ops.nn_ops import conv3d_v2 as conv3d 

221from tensorflow.python.ops.nn_ops import conv_transpose 

222from tensorflow.python.ops.nn_ops import convolution_v2 as convolution 

223from tensorflow.python.ops.nn_ops import crelu_v2 as crelu 

224from tensorflow.python.ops.nn_ops import depthwise_conv2d_native_backprop_filter as depthwise_conv2d_backprop_filter 

225from tensorflow.python.ops.nn_ops import depthwise_conv2d_native_backprop_input as depthwise_conv2d_backprop_input 

226from tensorflow.python.ops.nn_ops import dilation2d_v2 as dilation2d 

227from tensorflow.python.ops.nn_ops import dropout_v2 as dropout 

228from tensorflow.python.ops.nn_ops import erosion2d_v2 as erosion2d 

229from tensorflow.python.ops.nn_ops import fractional_avg_pool_v2 as fractional_avg_pool 

230from tensorflow.python.ops.nn_ops import fractional_max_pool_v2 as fractional_max_pool 

231from tensorflow.python.ops.nn_ops import gelu 

232from tensorflow.python.ops.nn_ops import in_top_k_v2 as in_top_k 

233from tensorflow.python.ops.nn_ops import isotonic_regression 

234from tensorflow.python.ops.nn_ops import leaky_relu 

235from tensorflow.python.ops.nn_ops import log_softmax_v2 as log_softmax 

236from tensorflow.python.ops.nn_ops import max_pool1d 

237from tensorflow.python.ops.nn_ops import max_pool2d 

238from tensorflow.python.ops.nn_ops import max_pool3d 

239from tensorflow.python.ops.nn_ops import max_pool_v2 as max_pool 

240from tensorflow.python.ops.nn_ops import max_pool_with_argmax_v2 as max_pool_with_argmax 

241from tensorflow.python.ops.nn_ops import pool_v2 as pool 

242from tensorflow.python.ops.nn_ops import relu6 

243from tensorflow.python.ops.nn_ops import softmax_cross_entropy_with_logits_v2 as softmax_cross_entropy_with_logits 

244from tensorflow.python.ops.nn_ops import softmax_v2 as softmax 

245from tensorflow.python.ops.nn_ops import sparse_softmax_cross_entropy_with_logits_v2 as sparse_softmax_cross_entropy_with_logits 

246from tensorflow.python.ops.nn_ops import top_k 

247from tensorflow.python.ops.nn_ops import with_space_to_batch