489 view times

function commonly used

view()

a = torch.arange(1, 17) # a’s shape is (16,)

a.view(4, 4) # output below

tensor([[ 1, 2, 3, 4],

[ 5, 6, 7, 8],

[ 9, 10, 11, 12],

[13, 14, 15, 16]])[torch.FloatTensor of size 4×4]

a.view(2, 2, 4) # output below

tensor([[[ 1, 2, 3, 4],

[ 5, 6, 7, 8]],

[[ 9, 10, 11, 12],

[13, 14, 15, 16]]])

[torch.FloatTensor of size 2x2x4]

this function like reshape() in numpy.

eval()

model.eval() fix the weight in the network. When testing the model, we do not want the parameter change. If we did not use this function, when the testset input into the model, the weight will change because of the BN layers.

bmm()

计算两个tensor的矩阵乘法,torch.bmm(a,b),tensor a 的size为(b,h,w),tensor b的size为(b,w,h),注意两个tensor的维度必须为3.

>>> cc=torch.randn((2,2,5))
>>>print(cc)
tensor([[[ 1.4873, -0.7482, -0.6734, -0.9682,  1.2869],
         [ 0.0550, -0.4461, -0.1102, -0.0797, -0.8349]],
         [[-0.6872,  1.1920, -0.9732,  0.4580,  0.7901],
         [ 0.3035,  0.2022,  0.8815,  0.9982, -1.1892]]])
>>>dd=torch.reshape(cc,(2,5,2))
>>> print(dd)
tensor([[[ 1.4873, -0.7482],
         [-0.6734, -0.9682],
         [ 1.2869,  0.0550],
         [-0.4461, -0.1102],
         [-0.0797, -0.8349]],
         [[-0.6872,  1.1920],
         [-0.9732,  0.4580],
         [ 0.7901,  0.3035],
         [ 0.2022,  0.8815],
         [ 0.9982, -1.1892]]])
>>>e=torch.bmm(cc,dd)
>>> print(e)
tensor([[[ 2.1787, -1.3931],
         [ 0.3425,  1.0906]],
         [[-0.5754, -1.1045],
         [-0.6941,  3.0161]]])
 >>> e.size()
torch.Size([2, 2, 2])


         

模型中需要保存下来的参数包括两种:

  • 一种是反向传播需要被optimizer更新的,称之为 parameter
  • 一种是反向传播不需要被optimizer更新,称之为 buffer

register_buffer()

register_buffer(name, tensor) name: string tensor: Tensor

register_parameter()

register_parameter(name, param) name: string param: Parameter

模型中需要进行更新的参数注册为Parameter,不需要进行更新的参数注册为buffer

nn.Parameter()

将一个不可训练的类型Tensor转换成可以训练的类型parameter,并绑定到对应的module里面,经过类型转换这个变量变成了模型的一部分,成为可以改动的参数,从而使得在迭代中优化

log1p(input, out=None)

说明:计算input+1的自然对数,对值比较小的输入,此函数比torch.log()更准确

参数

  • input(Tensor) — 输入张量
  • out(Tensor,可选) — 输出张量
>>> a = torch.randn(6)
>>> a
tensor([ 0.4177,  0.7744, -1.8840,  0.3302,  1.7383, -0.1667])
>>> torch.log1p(a)
tensor([ 0.3490,  0.5735,     nan,  0.2854,  1.0073, -0.1824])

clone()

返回一个和源张量同shapedtypedevice的张量,与源张量不共享数据内存,但提供梯度的回溯

detach()

detach的机制则与clone完全不同,即返回一个和源张量同shapedtypedevice的张量,与源张量共享数据内存,但不提供梯度计算,即requires_grad=False,因此脱离计算图。

copy_()

copy_同样将源张量中的数据复制到目标张量(数据不共享),其devicedtyperequires_grad一般都保留目标张量的设定,仅仅进行数据复制,同时其支持broadcast操作。

fill_(value) → Tensor

将该tensor用指定的数值填充

torch.split()

  • 第一个参数是待分割张量
  • 第二个参数有两种形式。
    一种是分割份数,这就和torch.chunk()一样了。
    第二种这是分割方案,这是一个list,待分割张量将会分割为len(list)份,每一份的大小取决于list中的元素
  • 第三个参数为分割维度
section=[1,2,1,2,2]
d=torch.randn(8,4)
print(torch.split(d,section,dim=0))
#输出结果为:
(tensor([[ 0.5388, -0.8537,  0.5539,  0.7793]]), tensor([[ 0.1270,  2.6241, -0.7594,  0.4644],
        [ 0.8160,  0.5553,  0.1234, -1.1157]]), tensor([[-0.4433, -0.3093, -2.0134, -0.4277]]), tensor([[-0.4297,  0.2532,  0.2789, -0.3068],
        [ 1.4208, -0.1202,  0.9256, -1.2127]]), tensor([[ 0.3542, -0.4656,  1.2683,  0.8753],
        [-0.2786, -0.2180,  0.3991,  0.5658]]))

contiguous()

返回一个内存连续的有相同数据的tensor,如果原tensor内存连续,则返回原tensor;

contiguous一般与transpose,permute,view搭配使用:使用transpose或permute进行维度变换后,调用contiguous,然后方可使用view对维度进行变形

原因:

1 transpose、permute等维度变换操作后,tensor在内存中不再是连续存储的,而view操作要求tensor的内存连续存储,所以需要contiguous来返回一个contiguous copy;

2 维度变换后的变量是之前变量的浅拷贝,指向同一区域,即view操作会连带原来的变量一同变形,这是不合法的,所以也会报错;—- 这个解释有部分道理,也即contiguous返回了tensor的深拷贝contiguous copy数据;

interpolate()

上采样下采样操作

def interpolate(input, size=None, scale_factor=None, mode='nearest', align_corners=None):
    r"""
    根据给定 size 或 scale_factor,上采样或下采样输入数据input.
    
    当前支持 temporal, spatial 和 volumetric 输入数据的上采样,其shape 分别为:3-D, 4-D 和 5-D.
    输入数据的形式为:mini-batch x channels x [optional depth] x [optional height] x width.

    上采样算法有:nearest, linear(3D-only), bilinear(4D-only), trilinear(5D-only).
    
    参数:
    - input (Tensor): input tensor
    - size (int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int]):输出的 spatial 尺寸.
    - scale_factor (float or Tuple[float]): spatial 尺寸的缩放因子.
    - mode (string): 上采样算法:nearest, linear, bilinear, trilinear, area. 默认为 nearest.
    - align_corners (bool, optional): 如果 align_corners=True,则对齐 input 和 output 的角点像素(corner pixels),保持在角点像素的值. 只会对 mode=linear, bilinear 和 trilinear 有作用. 默认是 False.
    """
    from numbers import Integral
    from .modules.utils import _ntuple

    def _check_size_scale_factor(dim):
        if size is None and scale_factor is None:
            raise ValueError('either size or scale_factor should be defined')
        if size is not None and scale_factor is not None:
            raise ValueError('only one of size or scale_factor should be defined')
        if scale_factor is not None and isinstance(scale_factor, tuple)\
                and len(scale_factor) != dim:
            raise ValueError('scale_factor shape must match input shape. '
                             'Input is {}D, scale_factor size is {}'.format(dim, len(scale_factor)))

    def _output_size(dim):
        _check_size_scale_factor(dim)
        if size is not None:
            return size
        scale_factors = _ntuple(dim)(scale_factor)
        # math.floor might return float in py2.7
        return [int(math.floor(input.size(i + 2) * scale_factors[i])) for i in range(dim)]

    if mode in ('nearest', 'area'):
        if align_corners is not None:
            raise ValueError("align_corners option can only be set with the "
                             "interpolating modes: linear | bilinear | trilinear")
    else:
        if align_corners is None:
            warnings.warn("Default upsampling behavior when mode={} is changed "
                          "to align_corners=False since 0.4.0. Please specify "
                          "align_corners=True if the old behavior is desired. "
                          "See the documentation of nn.Upsample for details.".format(mode))
            align_corners = False

    if input.dim() == 3 and mode == 'nearest':
        return torch._C._nn.upsample_nearest1d(input, _output_size(1))
    elif input.dim() == 4 and mode == 'nearest':
        return torch._C._nn.upsample_nearest2d(input, _output_size(2))
    elif input.dim() == 5 and mode == 'nearest':
        return torch._C._nn.upsample_nearest3d(input, _output_size(3))
    elif input.dim() == 3 and mode == 'area':
        return adaptive_avg_pool1d(input, _output_size(1))
    elif input.dim() == 4 and mode == 'area':
        return adaptive_avg_pool2d(input, _output_size(2))
    elif input.dim() == 5 and mode == 'area':
        return adaptive_avg_pool3d(input, _output_size(3))
    elif input.dim() == 3 and mode == 'linear':
        return torch._C._nn.upsample_linear1d(input, _output_size(1), align_corners)
    elif input.dim() == 3 and mode == 'bilinear':
        raise NotImplementedError("Got 3D input, but bilinear mode needs 4D input")
    elif input.dim() == 3 and mode == 'trilinear':
        raise NotImplementedError("Got 3D input, but trilinear mode needs 5D input")
    elif input.dim() == 4 and mode == 'linear':
        raise NotImplementedError("Got 4D input, but linear mode needs 3D input")
    elif input.dim() == 4 and mode == 'bilinear':
        return torch._C._nn.upsample_bilinear2d(input, _output_size(2), align_corners)
    elif input.dim() == 4 and mode == 'trilinear':
        raise NotImplementedError("Got 4D input, but trilinear mode needs 5D input")
    elif input.dim() == 5 and mode == 'linear':
        raise NotImplementedError("Got 5D input, but linear mode needs 3D input")
    elif input.dim() == 5 and mode == 'bilinear':
        raise NotImplementedError("Got 5D input, but bilinear mode needs 4D input")
    elif input.dim() == 5 and mode == 'trilinear':
        return torch._C._nn.upsample_trilinear3d(input, _output_size(3), align_corners)
    else:
        raise NotImplementedError("Input Error: Only 3D, 4D and 5D input Tensors supported"
                                  " (got {}D) for the modes: nearest | linear | bilinear | trilinear"
                                  " (got {})".format(input.dim(), mode))


    x = Variable(torch.randn([1, 3, 64, 64]))
    y0 = F.interpolate(x, scale_factor=0.5)
    y1 = F.interpolate(x, size=[32, 32])

    y2 = F.interpolate(x, size=[128, 128], mode="bilinear")

    print(y0.shape)
    print(y1.shape)
    print(y2.shape)

torch.Size([1, 3, 32, 32])
torch.Size([1, 3, 32, 32])
torch.Size([1, 3, 128, 128])

einsum

    x = torch.einsum('nkctv,kvw->nctw', (x, A))
#einstein sum, a convenient and flexible representation of various calculation~ actually, the code above is matrix outproduct

发表评论

邮箱地址不会被公开。 必填项已用*标注