python Archives - Redino blog

ImportError: cannot import name ‘pad’ from ‘skimage.util’

gbstack01/23/2022 0 Comments

I got following error after upgrading skimage package

Traceback (most recent call last):
  File "afy/yanderify.py", line 21, in <module>
    from afy.predictor_local import PredictorLocal
  File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\afy\predictor_local.py", line 9, in <module>
    from animate import normalize_kp
  File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\fomm\animate.py", line 7, in <module>
    from frames_dataset import PairedDataset
  File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\fomm\frames_dataset.py", line 10, in <module>
    from augmentation import AllAugmentationTransform
  File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\fomm\augmentation.py", line 12, in <module>
    from skimage.util import pad
ImportError: cannot import name 'pad' from 'skimage.util' (L:\ProgramData\Anaconda3\envs\fomm\lib\site-packages\skimage\util\__init__.py)

1

2

3

4

5

6

7

8

9

10

11

12

Traceback (most recent call last):

File "afy/yanderify.py", line 21, in <module>

from afy.predictor_local import PredictorLocal

File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\afy\predictor_local.py", line 9, in <module>

from animate import normalize_kp

File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\fomm\animate.py", line 7, in <module>

from frames_dataset import PairedDataset

File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\fomm\frames_dataset.py", line 10, in <module>

from augmentation import AllAugmentationTransform

File "K:\QQData\519329064\FileRecv\yanderifier\yanderify\cuda100\afy\fomm\augmentation.py", line 12, in <module>

from skimage.util import pad

ImportError: cannot import name 'pad' from 'skimage.util' (L:\ProgramData\Anaconda3\envs\fomm\lib\site-packages\skimage\util\__init__.py)

It's because skimage has removed pad API starting from version 0.18 (https://github.com/scikit-image/scikit-image/issues/4147)

Solution

So we should use np.pad instead of skimage.util.pad, and this issue will be fixed.

RuntimeError: view size is not compatible with input tensor’s size and stride (at least one dimension spans across two contiguous subspaces)

gbstack05/14/2021 0 Comments

RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

The causing code is

def accuracy(output, target, topk=(1,)):
    """Computes the accuracy over the k top predictions for the specified values of k"""
    with torch.no_grad():
        maxk = max(topk)
        batch_size = target.size(0)

        _, pred = output.topk(maxk, 1, True, True)
        pred = pred.t()
        correct = pred.eq(target.view(1, -1).expand_as(pred))

        res = []
        for k in topk:
            correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)
            res.append(correct_k.mul_(100.0 / batch_size))
        return res

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

def accuracy(output, target, topk=(1,)):

"""Computes the accuracy over the k top predictions for the specified values of k"""

with torch.no_grad():

maxk = max(topk)

batch_size = target.size(0)

_, pred = output.topk(maxk, 1, True, True)

pred = pred.t()

correct = pred.eq(target.view(1, -1).expand_as(pred))

res = []

for k in topk:

correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)

res.append(correct_k.mul_(100.0 / batch_size))

return res

This error didn't happen before, so it's introduced by new version of pytorch (now my current used pytorch is 1.8.1).

And printing the array, I found it's a boolean array.

Solution

Add .contiguous() before view() or use reshape to replace view

So change the line

correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)

1	correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)

to

correct_k = correct[:k].contiguous().view(-1).float().sum(0, keepdim=True)

1	correct_k = correct[:k].contiguous().view(-1).float().sum(0, keepdim=True)

or

correct_k = correct[:k].reshape(-1).float().sum(0, keepdim=True)

1	correct_k = correct[:k].reshape(-1).float().sum(0, keepdim=True)

Automatically generate requirements.txt for pip

gbstack12/18/2020 0 Comments

pip is popular package manager for Python.

There is often a requirements.txt file in lots of Python projects specifying several dependencies need to install.
We can install several packages using pip at once using following command

pip -r requirements.txt

to generate requirements for pip, certainly we can type the required package names line by line,
but there is a tool called pipreqs can help us.

pipreqs

pipreqs can analyze your python project for dependencies and generate requirements.txt
used by pip automatically.

Installation

pip install pipreqs

Generate requirements.txt

pipreqs .

This command will generate requirements.txt using current directory as source root directory.
Then you can run pip -r requirements.txt to install dependencies.

Python TypeError: Error when calling the metaclass bases

gbstack07/02/2017 0 Comments

When running the code, I got following error:

TypeError: Error when calling the metaclass bases
module.__init__() takes at most 2 arguments (3 given)

The source code is like following:

AboutWindow.py

import GitFtpWindow

class AboutWindow(GitFtpWindow):
    ...

1

2

3

4

import GitFtpWindow

class AboutWindow(GitFtpWindow):

...

GitFtpWindow.py

from PySide.QtGui import *

class GitFtpWindow(QWidget):
   def __init__(self):
      QWidget.__init__(self)
      self.setWindowTitle('Ftp-Git')

1

2

3

4

5

6

from PySide.QtGui import *

class GitFtpWindow(QWidget):

def __init__(self):

QWidget.__init__(self)

self.setWindowTitle('Ftp-Git')

This is because in AboutWindow.py file, the GitFtpWindow identifier is a package, but not a class.

To fix this problem, we need modify AboutWindow.py file like following:

from GitFtpWindow import GitFtpWindow

class AboutWindow(GitFtpWindow):

1

2

3

from GitFtpWindow import GitFtpWindow

class AboutWindow(GitFtpWindow):

No module named pefile

gbstack07/31/2016 1 Comment

Today I want to build a python script to EXE file using PyInstaller. But after running PyInstaller, I got following error

No module named pefile

The full stack trace is

Traceback (most recent call last):
  File "D:\\PyInstaller\\pyinstaller.py", line 14, in <module>
    from PyInstaller.__main__ import run
  File "D:\PyInstaller\PyInstaller\__main__.py", line 21, in <module>
    import PyInstaller.building.build_main
  File "D:\PyInstaller\PyInstaller\building\build_main.py", line 34, in <module>

    from .api import PYZ, EXE, COLLECT, MERGE
  File "D:\PyInstaller\PyInstaller\building\api.py", line 38, in <module>
    from PyInstaller.utils.win32 import winmanifest, icon, versioninfo, winresource
  File "D:\PyInstaller\PyInstaller\utils\win32\versioninfo.py", line 17, in <module>
    import pefile
ImportError: No module named pefile

1

2

3

4

5

6

7

8

9

10

11

12

13

Traceback (most recent call last):

File "D:\\PyInstaller\\pyinstaller.py", line 14, in <module>

from PyInstaller.__main__ import run

File "D:\PyInstaller\PyInstaller\__main__.py", line 21, in <module>

import PyInstaller.building.build_main

File "D:\PyInstaller\PyInstaller\building\build_main.py", line 34, in <module>

from .api import PYZ, EXE, COLLECT, MERGE

File "D:\PyInstaller\PyInstaller\building\api.py", line 38, in <module>

from PyInstaller.utils.win32 import winmanifest, icon, versioninfo, winresource

File "D:\PyInstaller\PyInstaller\utils\win32\versioninfo.py", line 17, in <module>

import pefile

ImportError: No module named pefile

Starting from PyInstaller 3.2, a new module named pefile is introduced. This module is used to work with Windows binary file (EXE file, also called PE file). Before PyInstaller 3.2, PE related operation is done using its own module, like setting PE header and DOS header)

Solution

Assume the Python installation direcotry is C:\Python27.

Open Command Prompt window, and navigate to C:\Python27\Scripts

cd C:\Python27\Scripts

Then install pefile module using following command

pip install pefile

(pip is built into latest Python distribution, 2.7.12 for now)

Multiprocessing for Frozen Python

gbstack04/11/2016 0 Comments

Recently I built a GUI application using PySide, and do some background calculation using multiprocessing. Using Python interpreter to run this application works smoothly. But if I freeze it to Windows executable file (exe) and run it, guess what, I got two main windows showing up!

After searching a while, I found following code should be added

multiprocessing.freeze_support()

1	multiprocessing.freeze_support()

It should be placed right after:

if __name__ == '__main__':

1	if __name__ == '__main__':

The freeze_support() is mainly used to pass initialization data from parent process to newly created process using pipe, including modules, process name, current working directory, etc. In Unix-based system this function is not needed, because fork will do these things for us (multiprocessing.Process will call os.fork() in start method on Linux)

the low-level implementation is in Lib/multiprocessing/forking.py and a wrapper in Lib/multiprocessing/__init__.py.

By reading the source code, we can see Python will detect --multiprocessing-fork in command line arguments to determine whether current process is child process or not. And the last command line argument is the pipe file handle. The data in main process is serialized using pickle, then pass to child process using pipe.

Memory Forensic for DateTime Type

gbstack02/26/2016 0 Comments

In C or C++, a DateTime value is often represented by time_t type, it's a UNIX timestamp format, which is number of seconds elapsed since 1970-1-1. This type is defined as long (4 bytes long) on 32 bit machine, and long long (8 bytes long) on 64 bit machine.

And Datetime value has one important trait:

In short time the high byte won't change.

Suppose a DateTime value is DDCCBBAA, so in memory it will be AA BB CC DD (in little endian), the highest byte (DD) won't change in short time, and the second highest byte (CC) don't change much. This is because the low 2 bytes can represent 65536 seconds, which is about 18 hours, that means after every 18 hours the second highest byte will only increase by 1. And the highest byte only change after 18*256 hours (nearly 192 days) passed by.

Next we will see how to use Python to convert bytes array to timestamp value, and display it as readable format

import struct, datetime

timestamp = struct.unpack('<L', data[4:8])[0]

print datetime.datetime.fromtimestamp(timestamp).strftime('%Y-%m-%d %H:%M:%S')

1

2

3

4

5

import struct, datetime

timestamp = struct.unpack('<L', data[4:8])[0]

print datetime.datetime.fromtimestamp(timestamp).strftime('%Y-%m-%d %H:%M:%S')

The data variable contains bytes needed work on, after some observing and investigation, we think byte 4 to byte 8 is the DateTime value. Then we use struct.unpack function to convert the bytes to timestamp value, '<L' means little endian and long type. Next we will convert timestamp value to human-readable format.

Note that time_t doesn't contain timezone information, you need discover timezone at other places.

PySide message box

gbstack04/28/2014 0 Comments

When you want to display a warning message (like "File doesn't exist") to user, using message box is a good choice. Following code is how to create a message box in PySide:

msg_box = QMessageBox()

msg_box.setText('Warning')
msgBox.setStandardButtons(QMessageBox.Ok|QMessageBox.Cancel)
msg_box.exec_()

1

2

3

4

5

msg_box = QMessageBox()

msg_box.setText('Warning')

msgBox.setStandardButtons(QMessageBox.Ok|QMessageBox.Cancel)

msg_box.exec_()

QMessageBox is the message window class, it's defined in PySide.QtGui module. setText method will set the information text, and exec_ method will display the message box and waiting for user's operation.

Python CGI setup (for Hostgator hosting)

gbstack03/08/2014 4 Comments

In this post we will enable Python CGI for Hostgator

Create a .htaccess file and put following contents in this file:

AddHandler cgi-script .py

1	AddHandler cgi-script .py

This will make the web server to interprete .py file as Python CGI script.

Next create a python file (named test.py here) with following code:

#!/usr/bin/python

print 'Content-Type: text/html\n\n'

print 'hello'

1

2

3

4

5

#!/usr/bin/python

print 'Content-Type: text/html\n\n'

print 'hello'

And set this file's permission setting so that Owner, Group and Public have "execute" permission for this file (Or set 755 permission simply)

Visit http://server_address/app_path/test.py, you will see a page with "hello" printed.

Note

The first two lines of code is necessary, otherwise you will get a "500 Internal Server Error".

python timer tutorial

gbstack02/01/2014 0 Comments

There are two methods in Python to make the process wait some seconds:

threading.Event

import threading

e = threading.Event()

e.wait(5)  #make the process to wait for 5 seconds

1

2

3

4

5

import threading

e = threading.Event()

e.wait(5) #make the process to wait for 5 seconds

time.sleep

import time

time.sleep(2)  #let the program to sleep for 2 seconds

1

2

3

import time

time.sleep(2) #let the program to sleep for 2 seconds