Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/nn/__init__.py: 100%

1# This file is MACHINE GENERATED! Do not edit. 

2# Generated by: tensorflow/python/tools/api/generator/create_python_api.py script. 

3"""Primitive Neural Net (NN) Operations. 

4 

5## Notes on padding 

6 

7Several neural network operations, such as `tf.nn.conv2d` and 

8`tf.nn.max_pool2d`, take a `padding` parameter, which controls how the input is 

9padded before running the operation. The input is padded by inserting values 

10(typically zeros) before and after the tensor in each spatial dimension. The 

11`padding` parameter can either be the string `'VALID'`, which means use no 

12padding, or `'SAME'` which adds padding according to a formula which is 

13described below. Certain ops also allow the amount of padding per dimension to 

14be explicitly specified by passing a list to `padding`. 

15 

16In the case of convolutions, the input is padded with zeros. In case of pools, 

17the padded input values are ignored. For example, in a max pool, the sliding 

18window ignores padded values, which is equivalent to the padded values being 

19`-infinity`. 

20 

21### `'VALID'` padding 

22 

23Passing `padding='VALID'` to an op causes no padding to be used. This causes the 

24output size to typically be smaller than the input size, even when the stride is 

25one. In the 2D case, the output size is computed as: 

26 

27```python 

28out_height = ceil((in_height - filter_height + 1) / stride_height) 

29out_width = ceil((in_width - filter_width + 1) / stride_width) 

30``` 

31 

32The 1D and 3D cases are similar. Note `filter_height` and `filter_width` refer 

33to the filter size after dilations (if any) for convolutions, and refer to the 

34window size for pools. 

35 

36### `'SAME'` padding 

37 

38With `'SAME'` padding, padding is applied to each spatial dimension. When the 

39strides are 1, the input is padded such that the output size is the same as the 

40input size. In the 2D case, the output size is computed as: 

41 

42```python 

43out_height = ceil(in_height / stride_height) 

44out_width = ceil(in_width / stride_width) 

45``` 

46 

47The amount of padding used is the smallest amount that results in the output 

48size. The formula for the total amount of padding per dimension is: 

49 

50```python 

51if (in_height % strides[1] == 0): 

52 pad_along_height = max(filter_height - stride_height, 0) 

53else: 

54 pad_along_height = max(filter_height - (in_height % stride_height), 0) 

55if (in_width % strides[2] == 0): 

56 pad_along_width = max(filter_width - stride_width, 0) 

57else: 

58 pad_along_width = max(filter_width - (in_width % stride_width), 0) 

59``` 

60 

61Finally, the padding on the top, bottom, left and right are: 

62 

63```python 

64pad_top = pad_along_height // 2 

65pad_bottom = pad_along_height - pad_top 

66pad_left = pad_along_width // 2 

67pad_right = pad_along_width - pad_left 

68``` 

69 

70Note that the division by 2 means that there might be cases when the padding on 

71both sides (top vs bottom, right vs left) are off by one. In this case, the 

72bottom and right sides always get the one additional padded pixel. For example, 

73when pad_along_height is 5, we pad 2 pixels at the top and 3 pixels at the 

74bottom. Note that this is different from existing libraries such as PyTorch and 

75Caffe, which explicitly specify the number of padded pixels and always pad the 

76same number of pixels on both sides. 

77 

78Here is an example of `'SAME'` padding: 

79 

80>>> in_height = 5 

81>>> filter_height = 3 

82>>> stride_height = 2 

83>>> 

84>>> in_width = 2 

85>>> filter_width = 2 

86>>> stride_width = 1 

87>>> 

88>>> inp = tf.ones((2, in_height, in_width, 2)) 

89>>> filter = tf.ones((filter_height, filter_width, 2, 2)) 

90>>> strides = [stride_height, stride_width] 

91>>> output = tf.nn.conv2d(inp, filter, strides, padding='SAME') 

92>>> output.shape[1] # output_height: ceil(5 / 2) 

933 

94>>> output.shape[2] # output_width: ceil(2 / 1) 

952 

96 

97### Explicit padding 

98 

99Certain ops, like `tf.nn.conv2d`, also allow a list of explicit padding amounts 

100to be passed to the `padding` parameter. This list is in the same format as what 

101is passed to `tf.pad`, except the padding must be a nested list, not a tensor. 

102For example, in the 2D case, the list is in the format `[[0, 0], [pad_top, 

103pad_bottom], [pad_left, pad_right], [0, 0]]` when `data_format` is its default 

104value of `'NHWC'`. The two `[0, 0]` pairs indicate the batch and channel 

105dimensions have no padding, which is required, as only spatial dimensions can 

106have padding. 

107 

108For example: 

109 

110>>> inp = tf.ones((1, 3, 3, 1)) 

111>>> filter = tf.ones((2, 2, 1, 1)) 

112>>> strides = [1, 1] 

113>>> padding = [[0, 0], [1, 2], [0, 1], [0, 0]] 

114>>> output = tf.nn.conv2d(inp, filter, strides, padding=padding) 

115>>> tuple(output.shape) 

116(1, 5, 3, 1) 

117>>> # Equivalently, tf.pad can be used, since convolutions pad with zeros. 

118>>> inp = tf.pad(inp, padding) 

119>>> # 'VALID' means to use no padding in conv2d (we already padded inp) 

120>>> output2 = tf.nn.conv2d(inp, filter, strides, padding='VALID') 

121>>> tf.debugging.assert_equal(output, output2) 

122 

123### Difference between convolution and pooling layers 

124How padding is used in convolution layers and pooling layers is different. For 

125convolution layers, padding is filled with values of zero, and padding is 

126multiplied with kernels. For pooling layers, padding is excluded from the 

127computation. For example when applying average pooling to a 4x4 grid, how much 

128padding is added will not impact the output. Here is an example that 

129demonstrates the difference. 

130 

131>>> x_in = np.array([[ 

132... [[2], [2]], 

133... [[1], [1]], 

134... [[1], [1]]]]) 

135>>> kernel_in = np.array([ # simulate the avg_pool with conv2d 

136... [ [[0.25]], [[0.25]] ], 

137... [ [[0.25]], [[0.25]] ]]) 

138>>> x = tf.constant(x_in, dtype=tf.float32) 

139>>> kernel = tf.constant(kernel_in, dtype=tf.float32) 

140>>> conv_out = tf.nn.conv2d(x, kernel, strides=[1, 1, 1, 1], padding='SAME') 

141>>> pool_out = tf.nn.avg_pool(x, [2, 2], strides=[1, 1, 1, 1], padding='SAME') 

142>>> print(conv_out.shape, pool_out.shape) 

143(1, 3, 2, 1) (1, 3, 2, 1) 

144>>> tf.reshape(conv_out, [3, 2]).numpy() # conv2d takes account of padding 

145array([[1.5 , 0.75], 

146 [1. , 0.5 ], 

147 [0.5 , 0.25]], dtype=float32) 

148>>> tf.reshape(pool_out, [3, 2]).numpy() # avg_pool excludes padding 

149array([[1.5, 1.5], 

150 [1. , 1. ], 

151 [1. , 1. ]], dtype=float32) 

152 

153 

154""" 

155 

156import sys as _sys 

157 

158from . import experimental 

159from . import rnn_cell 

160from tensorflow.python.ops.array_ops import depth_to_space 

161from tensorflow.python.ops.array_ops import space_to_batch 

162from tensorflow.python.ops.array_ops import space_to_depth 

163from tensorflow.python.ops.candidate_sampling_ops import all_candidate_sampler 

164from tensorflow.python.ops.candidate_sampling_ops import compute_accidental_hits 

165from tensorflow.python.ops.candidate_sampling_ops import fixed_unigram_candidate_sampler 

166from tensorflow.python.ops.candidate_sampling_ops import learned_unigram_candidate_sampler 

167from tensorflow.python.ops.candidate_sampling_ops import log_uniform_candidate_sampler 

168from tensorflow.python.ops.candidate_sampling_ops import uniform_candidate_sampler 

169from tensorflow.python.ops.ctc_ops import collapse_repeated 

170from tensorflow.python.ops.ctc_ops import ctc_beam_search_decoder 

171from tensorflow.python.ops.ctc_ops import ctc_beam_search_decoder_v2 

172from tensorflow.python.ops.ctc_ops import ctc_greedy_decoder 

173from tensorflow.python.ops.ctc_ops import ctc_loss 

174from tensorflow.python.ops.ctc_ops import ctc_loss_v2 

175from tensorflow.python.ops.ctc_ops import ctc_unique_labels 

176from tensorflow.python.ops.embedding_ops import embedding_lookup 

177from tensorflow.python.ops.embedding_ops import embedding_lookup_sparse 

178from tensorflow.python.ops.embedding_ops import safe_embedding_lookup_sparse 

179from tensorflow.python.ops.gen_math_ops import tanh 

180from tensorflow.python.ops.gen_nn_ops import conv3d_backprop_filter_v2 

181from tensorflow.python.ops.gen_nn_ops import conv3d_backprop_filter_v2 as conv3d_backprop_filter 

182from tensorflow.python.ops.gen_nn_ops import elu 

183from tensorflow.python.ops.gen_nn_ops import l2_loss 

184from tensorflow.python.ops.gen_nn_ops import lrn 

185from tensorflow.python.ops.gen_nn_ops import lrn as local_response_normalization 

186from tensorflow.python.ops.gen_nn_ops import quantized_avg_pool 

187from tensorflow.python.ops.gen_nn_ops import quantized_conv2d 

188from tensorflow.python.ops.gen_nn_ops import quantized_max_pool 

189from tensorflow.python.ops.gen_nn_ops import quantized_relu_x 

190from tensorflow.python.ops.gen_nn_ops import relu 

191from tensorflow.python.ops.gen_nn_ops import selu 

192from tensorflow.python.ops.gen_nn_ops import softsign 

193from tensorflow.python.ops.math_ops import sigmoid 

194from tensorflow.python.ops.math_ops import softplus 

195from tensorflow.python.ops.nn_impl import batch_norm_with_global_normalization 

196from tensorflow.python.ops.nn_impl import batch_normalization 

197from tensorflow.python.ops.nn_impl import compute_average_loss 

198from tensorflow.python.ops.nn_impl import depthwise_conv2d 

199from tensorflow.python.ops.nn_impl import fused_batch_norm 

200from tensorflow.python.ops.nn_impl import l2_normalize 

201from tensorflow.python.ops.nn_impl import log_poisson_loss 

202from tensorflow.python.ops.nn_impl import moments 

203from tensorflow.python.ops.nn_impl import nce_loss 

204from tensorflow.python.ops.nn_impl import normalize_moments 

205from tensorflow.python.ops.nn_impl import relu_layer 

206from tensorflow.python.ops.nn_impl import sampled_softmax_loss 

207from tensorflow.python.ops.nn_impl import scale_regularization_loss 

208from tensorflow.python.ops.nn_impl import separable_conv2d 

209from tensorflow.python.ops.nn_impl import sigmoid_cross_entropy_with_logits 

210from tensorflow.python.ops.nn_impl import sufficient_statistics 

211from tensorflow.python.ops.nn_impl import swish 

212from tensorflow.python.ops.nn_impl import swish as silu 

213from tensorflow.python.ops.nn_impl import weighted_cross_entropy_with_logits 

214from tensorflow.python.ops.nn_impl import weighted_moments 

215from tensorflow.python.ops.nn_impl import zero_fraction 

216from tensorflow.python.ops.nn_ops import approx_max_k 

217from tensorflow.python.ops.nn_ops import approx_min_k 

218from tensorflow.python.ops.nn_ops import atrous_conv2d 

219from tensorflow.python.ops.nn_ops import atrous_conv2d_transpose 

220from tensorflow.python.ops.nn_ops import avg_pool 

221from tensorflow.python.ops.nn_ops import avg_pool as avg_pool2d 

222from tensorflow.python.ops.nn_ops import avg_pool1d 

223from tensorflow.python.ops.nn_ops import avg_pool3d 

224from tensorflow.python.ops.nn_ops import avg_pool_v2 

225from tensorflow.python.ops.nn_ops import bias_add 

226from tensorflow.python.ops.nn_ops import conv1d 

227from tensorflow.python.ops.nn_ops import conv1d_transpose 

228from tensorflow.python.ops.nn_ops import conv2d 

229from tensorflow.python.ops.nn_ops import conv2d_backprop_filter 

230from tensorflow.python.ops.nn_ops import conv2d_backprop_input 

231from tensorflow.python.ops.nn_ops import conv2d_transpose 

232from tensorflow.python.ops.nn_ops import conv3d_transpose 

233from tensorflow.python.ops.nn_ops import conv3d_v1 as conv3d 

234from tensorflow.python.ops.nn_ops import conv_transpose 

235from tensorflow.python.ops.nn_ops import convolution 

236from tensorflow.python.ops.nn_ops import crelu 

237from tensorflow.python.ops.nn_ops import depthwise_conv2d_native 

238from tensorflow.python.ops.nn_ops import depthwise_conv2d_native_backprop_filter 

239from tensorflow.python.ops.nn_ops import depthwise_conv2d_native_backprop_filter as depthwise_conv2d_backprop_filter 

240from tensorflow.python.ops.nn_ops import depthwise_conv2d_native_backprop_input 

241from tensorflow.python.ops.nn_ops import depthwise_conv2d_native_backprop_input as depthwise_conv2d_backprop_input 

242from tensorflow.python.ops.nn_ops import dilation2d_v1 as dilation2d 

243from tensorflow.python.ops.nn_ops import dropout 

244from tensorflow.python.ops.nn_ops import erosion2d 

245from tensorflow.python.ops.nn_ops import fractional_avg_pool 

246from tensorflow.python.ops.nn_ops import fractional_max_pool 

247from tensorflow.python.ops.nn_ops import in_top_k 

248from tensorflow.python.ops.nn_ops import leaky_relu 

249from tensorflow.python.ops.nn_ops import log_softmax 

250from tensorflow.python.ops.nn_ops import max_pool 

251from tensorflow.python.ops.nn_ops import max_pool1d 

252from tensorflow.python.ops.nn_ops import max_pool2d 

253from tensorflow.python.ops.nn_ops import max_pool3d 

254from tensorflow.python.ops.nn_ops import max_pool_v2 

255from tensorflow.python.ops.nn_ops import max_pool_with_argmax_v1 as max_pool_with_argmax 

256from tensorflow.python.ops.nn_ops import pool 

257from tensorflow.python.ops.nn_ops import relu6 

258from tensorflow.python.ops.nn_ops import softmax 

259from tensorflow.python.ops.nn_ops import softmax_cross_entropy_with_logits 

260from tensorflow.python.ops.nn_ops import softmax_cross_entropy_with_logits_v2_helper as softmax_cross_entropy_with_logits_v2 

261from tensorflow.python.ops.nn_ops import sparse_softmax_cross_entropy_with_logits 

262from tensorflow.python.ops.nn_ops import top_k 

263from tensorflow.python.ops.nn_ops import with_space_to_batch 

264from tensorflow.python.ops.nn_ops import xw_plus_b 

265from tensorflow.python.ops.rnn import bidirectional_dynamic_rnn 

266from tensorflow.python.ops.rnn import dynamic_rnn 

267from tensorflow.python.ops.rnn import raw_rnn 

268from tensorflow.python.ops.rnn import static_bidirectional_rnn 

269from tensorflow.python.ops.rnn import static_rnn 

270from tensorflow.python.ops.rnn import static_state_saving_rnn