Expected string or bytes like object python ошибка

In this article we are going to see what causes the error named typeerror: expected string or bytes like object. Basically, we will encounter this type of error, when we are working with a regular expression and we are trying to replace any value with some other value.

Whenever we are working with our code to develop something and we write some code, then it may be possible our code encounters any type of error. One of those error is Typeerror, which indicates that we have not given the type of data required. For example, if we try to add 3 + “five”, then we will encounter an error named typeerror because the summation between an integer and a string is not supported in python.

Whenever this type of error occurs, we have to make sure that we are passing the same type of data that is required to complete the operation.

So, in the above sum we are required to pass an integer to add in 3.

It will eliminate our error and the operation will be successfully performed.

In this article we are going to see what causes the error named typeerror: expected string or bytes like object. Basically, we will encounter this type of error, when we are working with a regular expression and we are trying to replace any value with some other value.

Example 1:

import pandas as pd

import re

data = pd.DataFrame({‘Name’:[«Alpha», «Beta», 12, «Gamma»]})

print(data)

data[‘Name’] = [re.sub(str(i),«Omega», i) if i == 12 else i for i in                data[‘Name’]]

print(data)

Output:

PS C: Users ASUS Desktop Crazy Programmer Work > python u » c : Users ASUS Desktop Crazy Programmer Work test.py «

      Name

0  Alpha

1   Beta

2     12

3  Gamma

Traceback (most recent call last):

  File «c : Users ASUS Desktop Crazy Programmer Work test.py «,  line 8, in <module>

    data[‘Name’] = [re.sub(str(i),«Omega», i) if i == 12 else i for i in data[‘Name’]]

  File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 8, in <listcomp>

    data[‘Name’] = [re.sub(str(i),«Omega», i) if i == 12 else i for i in data[‘Name’]]

  File «C : Program Files Python382 lib re.py», line 208, in sub

    return _compile(pattern, flags).sub(repl, string, count)

TypeError: expected string or byteslike object

Here we have a DataFrame in which there are some data. In the initial stage when we are trying to print it, it is printed successfully.

Let us suppose we want to change the value of 12 in the Name column.

When we are trying to change it, it is giving the same error as we have discussed earlier.

So we have to check our code that whether the bug is there. As I described earlier this error occurs when we are trying to give any other datatype value different from its normal type which is required in the operation.

Here we are using a regular expression to replace the integer value with a string value.

So, we have to make a simple change in our code to execute it successfully.

As we can see from the documentation of regular expression that re.sub() function required its third parameter as a string but in the code we have passes it as integers.

Solution:

To resolve it, we have to pass the third parameter named I as a string. We have to write str(i) instead of I to remove this error.

import pandas as pd

import re

data = pd.DataFrame({‘Name’:[«Alpha», «Beta», 12, «Gamma»]})

print(data)

data[‘Name’] = [re.sub(str(i),«Omega», str(i)) if i == 12 else i for i in data[‘Name’]]

print(data)

Output:

PS C: Users ASUS Desktop Crazy Programmer Work > python u » c : Users ASUS Desktop Crazy Programmer Work test.py «

    Name

0  Alpha

1   Beta

2     12

3  Gamma

    Name

0  Alpha

1   Beta

2  Omega

3  Gamma

As we can see that after passing the third parameter as string, we are successfully able to perform the desired operation using regular expression.

We have successfully changed the value of integer 12 to string Omega.

Example 2:

import re

list1 = [‘Alpha’, ‘Beta’, ‘Gamma’, 40, 30]

list1 = [re.sub(str(item), «this», item) if item in range(10, 41) else item for item in list1 ]

print(list1)

Output:

PS C: Users ASUS Desktop Crazy Programmer Work > python u » c : Users ASUS Desktop Crazy Programmer Work test.py «

Traceback (most recent call last):

  File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 59, in <module>

    list1 = [re.sub(str(item), «this», item) if item in range(10, 41) else item for item in list1 ]

  File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 59, in <listcomp>

    list1 = [re.sub(str(item), «this», item) if item in range(10, 41) else item for item in list1 ]

  File «C: Program Files Python382 lib re.py «, line 208, in sub

    return _compile(pattern, flags).sub(repl, string, count)

TypeError: expected string or byteslike object

In this example, we are trying to replace some values in the list which are integers ranging from 10 to 40. We are performing this operation using sub() function in a regular expression.

As we can see that we have encountered with the same error. Whenever we are working with sub() function, then we have to cross check that we are passing the correct argument type else we will encounter the same error and it is a little bit difficult to find where the exact error is.

As we can see that we are passing the third argument as an integer that why we have encountered the error.

Solution:

We can resolve this error by passing the third argument as string type by converting it into a string.

Let’s see the corrected code and its output.

import re

list1 = [‘Alpha’, ‘Beta’, ‘Gamma’, 40, 30]

list1 = [re.sub(str(item), «this», str(item)) if item in range(10, 41) else item for item in list1 ]

print(list1)

Output:

PS C: Users ASUS Desktop Crazy Programmer Work > python u » c : Users ASUS Desktop Crazy Programmer Work test.py «

[‘Alpha’, ‘Beta’, ‘Gamma’, ‘this’, ‘this’]

As we can see that after passing the third argument as a string, we have successfully fixed the error which was shown earlier.

Conclusion

This type of error mostly occurs when we are working with sub() function in a regular expression. This work of this function is to replace a given pattern with some string. We have to be careful when we are passing its arguments. We have to check which type of argument is required to perform the operation. Whenever we pass the argument of other type which is not required then we will encounter this type of error.

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

Labels

kind/bug

Something isn’t working as expected

Comments

@wichert

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: based on official docker image python:3.8-slim-buster (see below for Dockerfile)
  • Poetry version: 1.1.4
  • virtualenv version: 20.4.2
  • Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/wichert/ba05b0505c24ee7c71c5d7deb2ac2cfe

Issue

When I run poetry install in a private GitHub runner on an GHES installation I started getting an error (see further below for full output):

  TypeError

  expected string or bytes-like object

  at /usr/local/lib/python3.8/site-packages/poetry/core/utils/helpers.py:24 in canonicalize_name

Debug output

Creating virtualenv amp-portal in /__w/backend/backend/amp-portal/.venv
Using virtualenv: /__w/backend/backend/amp-portal/.venv

  Stack trace:

  11  /usr/local/lib/python3.8/site-packages/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│ 
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  10  /usr/local/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

   9  /usr/local/lib/python3.8/site-packages/clikit/api/command/command.py:163 in _do_handle
       161│         if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
       162│             event = PreHandleEvent(args, io, self)
     → 163│             self._dispatcher.dispatch(PRE_HANDLE, event)
       164│ 
       165│             if event.is_handled():

   8  /usr/local/lib/python3.8/site-packages/clikit/api/event/event_dispatcher.py:22 in dispatch
        20│ 
        21│         if listeners:
     →  22│             self._do_dispatch(listeners, event_name, event)
        23│ 
        24│         return event

   7  /usr/local/lib/python3.8/site-packages/clikit/api/event/event_dispatcher.py:89 in _do_dispatch
        87│                 break
        88│ 
     →  89│             listener(event, event_name, self)
        90│ 
        91│     def _sort_listeners(self, event_name):  # type: (str) -> None

   6  /usr/local/lib/python3.8/site-packages/poetry/console/config/application_config.py:141 in set_installer
       139│ 
       140│         poetry = command.poetry
     → 141│         installer = Installer(
       142│             event.io,
       143│             command.env,

   5  /usr/local/lib/python3.8/site-packages/poetry/installation/installer.py:65 in __init__
        63│         self._installer = self._get_installer()
        64│         if installed is None:
     →  65│             installed = self._get_installed()
        66│ 
        67│         self._installed_repository = installed

   4  /usr/local/lib/python3.8/site-packages/poetry/installation/installer.py:561 in _get_installed
       559│ 
       560│     def _get_installed(self):  # type: () -> InstalledRepository
     → 561│         return InstalledRepository.load(self._env)
       562│ 

   3  /usr/local/lib/python3.8/site-packages/poetry/repositories/installed_repository.py:118 in load
       116│                 path = Path(str(distribution._path))
       117│                 version = distribution.metadata["version"]
     → 118│                 package = Package(name, version, version)
       119│                 package.description = distribution.metadata.get("summary", "")
       120│ 

   2  /usr/local/lib/python3.8/site-packages/poetry/core/packages/package.py:51 in __init__
        49│         Creates a new in memory package.
        50│         """
     →  51│         super(Package, self).__init__(
        52│             name,
        53│             source_type=source_type,

   1  /usr/local/lib/python3.8/site-packages/poetry/core/packages/specification.py:19 in __init__
        17│     ):  # type: (str, Optional[str], Optional[str], Optional[str], Optional[str], Optional[List[str]]) -> None
        18│         self._pretty_name = name
     →  19│         self._name = canonicalize_name(name)
        20│         self._source_type = source_type
        21│         self._source_url = source_url

  TypeError

  expected string or bytes-like object

  at /usr/local/lib/python3.8/site-packages/poetry/core/utils/helpers.py:24 in canonicalize_name
       20│ _canonicalize_regex = re.compile("[-_]+")
       21│ 
       22│ 
       23│ def canonicalize_name(name):  # type: (str) -> str
    →  24│     return _canonicalize_regex.sub("-", name).lower()
       25│ 
       26│ 
       27│ def module_name(name):  # type: (str) -> str
       28│     return canonicalize_name(name).replace(".", "_").replace("-", "_")

Dockerfile for base docker image

ARG DEBIAN_VERSION=buster
ARG PYTHON_VERSION=3.8
FROM python:${PYTHON_VERSION}-slim-${DEBIAN_VERSION}

ENV 
    DEBIAN_FRONTEND=noninteractive 
    PYTHONUNBUFFERED=1 
    PYTHONDONTWRITEBYTECODE=1 
    POETRY_VIRTUALENVS_IN_PROJECT=true 
    POETRY_NO_INTERACTION=1

RUN set -x 
    && apt-get update 
    && apt-get install -yq --no-install-recommends libpq-dev build-essential sudo 
    && rm -rf /var/lib/apt/lists/*

RUN pip install poetry==1.1.4
MehdiABM, NawfalTachfine2, remram44, ErikBjare, Saelyos, hleb-albau, brylie, golmschenk, EmilLaursen, stahlnow, and 17 more reacted with thumbs up emoji

@sinoroc

@sinoroc

We should try to narrow it down, maybe there is a dependency somewhere with packaging issues.

@wichert

Some extra information:

  • we use a GHES installation with multiple runners. On some runners it fails every time, on others it always succeeds.
  • this is not dependent on the local .venv directory: that is newly created on every run
  • it is not dependent on what is in the users home directory: that is part of the docker container and is identical on all runners.

So there is something very specific to a runner that is independent from the docker image that triggers this error.

@sinoroc

I see. This is good info. This changes my perspective on this issue.

@wichert

Perhaps this could be a broken cache entry somewhere?

@wichert

I’m adding @netcaptors here; he is a member of the team that manages the github runners and has more insight into their configuration than I have.

@sinoroc

I’m out of my depth on this. I’ll ping @stephsamson and @finswimmer maybe they’ll be able to give more insight or redirect to other maintainers.

But you seem to say it’s rather new behaviour, although there was no new poetry release in a while, right?

@wichert

This particularly error is new for us today as far as I can tell. I expect that there is something in the environment that poetry needs to deal with that changed and triggered this error.

@sinoroc

@wichert

Doubtful, we had not done a rebuild of our docker image at the moment this error started occurring as far as I know.

@wichert

As magically as this error started appearing, it seems to have disappeared today. As far as I know the only change that happened is that chmod -R github-runner:github-runner /home/github-runner/runner/_work/ was run on the runner instance that was showing the error. The best theory I have is that there was a directory or file somewhere that was not readable by the user, which triggered this error. Perhaps a .dist-info directory that was not readable?

@niyue

I saw this issue locally today on my mac. I used poetry 1.1.4 and it worked for me until today, for some reason, poetry install doesn’t work and reported the same error in this issue. Commands like poetry show stops working too with the same error.

I tried revert everything to a specific git commit I confirmed working recently (delete the entire folder and fetch from git again, so it doesn’t seem to me a permission issue), but the same error was reported.

@MaikuMori

I got the same problem. I dropped into PDB and tried to track it down.

name is None and it comes from here:

https://github.com/python-poetry/poetry/blob/1.1.4/poetry/repositories/installed_repository.py#L115

Then if I run this in PDB:
pp [x.metadata["Name"] for x in metadata.distributions(path=[env.sys_path[4]])]

I get for my specific .venv/project I get something like:

[ '...',
 'dbus-python',
 None,
 'pluggy',
 'flake8',
 '...']

Note the None.

Now let’s find the culprit:
pp [x.files if x.metadata["name"] is None else "SKIP" for x in metadata.distributions(path=[env.sys_path[4]])]

[ '...',
 'SKIP',
 [PackagePath('more_itertools-8.7.0.dist-info/INSTALLER'),
  PackagePath('more_itertools-8.7.0.dist-info/LICENSE'),
  PackagePath('more_itertools-8.7.0.dist-info/METADATA'),
  PackagePath('more_itertools-8.7.0.dist-info/RECORD'),
  PackagePath('more_itertools-8.7.0.dist-info/REQUESTED'),
  PackagePath('more_itertools-8.7.0.dist-info/WHEEL'),
  PackagePath('more_itertools-8.7.0.dist-info/direct_url.json'),
  PackagePath('more_itertools-8.7.0.dist-info/top_level.txt'),
  PackagePath('more_itertools/__init__.py'),
  PackagePath('more_itertools/__init__.pyi'),
  PackagePath('more_itertools/__pycache__/__init__.cpython-39.pyc'),
  PackagePath('more_itertools/__pycache__/more.cpython-39.pyc'),
  PackagePath('more_itertools/__pycache__/recipes.cpython-39.pyc'),
  PackagePath('more_itertools/more.py'),
  PackagePath('more_itertools/more.pyi'),
  PackagePath('more_itertools/py.typed'),
  PackagePath('more_itertools/recipes.py'),
  PackagePath('more_itertools/recipes.pyi')],
 'SKIP',
 '...']

For some reason PathDistribution object representing more_itertools package has empty metadata and version fields.

Interestingly enough it’s also the only dependency of my project dependencies which upgraded when I last ran poetry update. Not sure if that’s related, or they broke something.

Hope this helps.

EDIT:
I actually checked the METADATA file in more_itertools-8.7.0.dist-info directory and it’s empty.

Extra info:
pip version in venv: 21.0.1
pip version outside: 20.3.3
poetry version: 1.1.4

EDIT 2:
I bet it has something to do with pip version as mentioned before.

EDIT 3:
I ended up having to clear ~/.cache/pypoetry seems like the whole artifact was corrupted. Without that even deleting and reinstalling .venv would crash on poetry install.

@tulis

I was also having the same problem.

In my case, I ran poetry shell to test out running pyInstaller command inside the shell. Once I exited the shell and tried to run command poetry run or poetry install, I got expected string or bytes-like object error message.

MaikuMori’s third solution, removing ~/.cache/pypoetry did the trick and solved the issue.

Update:
Removing ~/.cache/pypoetry did work temporarily. The first command either poetry install or poetry run will work, but on subsequent command still got the same error message. To get around it, remove the cache then run poetry command again and keeps repeating the cycle.

lazToum

added a commit
to lazToum/poetry
that referenced
this issue

Mar 21, 2021

@lazToum

@ElricleNecro

We also get those expected string or bytes-like object a lot, but we are working in a conda environment on MacOS. For us, it seems to happen more often when we are changing branch in our soft repo, then doing a poetry install, or when updating a dependency to use the path directive ('toto' = {path='<some-path>', develop=true}), for version. Unfortunately, I didn’t find a way to consistently reproduce those errors.

@kitmonisit

First, I start with a pyenv-installed python-3.8.8:

$ pyenv install 3.8.8
$ pyenv global 3.8.8

I try to reproduce the problem after a fresh installation of poetry-1.1.5:

$ mkdir debug
$ cd debug
$ poetry new somepackage
$ cd somepackage
$ poetry install # This creates the virtual environment in ~/.cache/pypoetry/virtualenvs

  TypeError

  expected string or bytes-like object

  at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/utils/helpers.py:27 in canonicalize_name
       23│ _canonicalize_regex = re.compile(r"[-_]+")
       24│ 
       25│ 
       26│ def canonicalize_name(name):  # type: (str) -> str
    →  27│     return _canonicalize_regex.sub("-", name).lower()
       28│ 
       29│ 
       30│ def module_name(name):  # type: (str) -> str
       31│     return canonicalize_name(name).replace(".", "_").replace("-", "_")

Now, I inspect the newly-created virtual environment’s site-packages

$ cd ~/.cache/pypoetry/virtualenvs/somepackage-RAiBNyBV-py3.8/lib/python3.8/site-packages
$ ls -al

total 44K
drwxr-xr-x 2 myuser mygrp 4.0K Apr  7 09:23 _distutils_hack
drwxr-xr-x 4 myuser mygrp 4.0K Apr  7 09:23 pip
drwxr-xr-x 5 myuser mygrp 4.0K Apr  7 09:23 pkg_resources
drwxr-xr-x 2 myuser mygrp 4.0K Apr  7 09:23 __pycache__
drwxr-xr-x 6 myuser mygrp 4.0K Apr  7 09:23 setuptools
drwxr-xr-x 4 myuser mygrp 4.0K Apr  7 09:23 wheel
drwxr-xr-x 2 myuser mygrp 4.0K Apr  7 09:23 pip-21.0.1.dist-info
drwxr-xr-x 2 myuser mygrp 4.0K Apr  7 09:23 setuptools-52.0.0.dist-info
drwxr-xr-x 2 myuser mygrp 4.0K Apr  7 09:23 wheel-0.36.2.dist-info
-rw-r--r-- 1 myuser mygrp    0 Apr  7 09:23 distutils-precedence.pth
-rw-r--r-- 1 myuser mygrp   18 Apr  7 09:23 _virtualenv.pth
-rw-r--r-- 1 myuser mygrp 5.6K Apr  7 09:23 _virtualenv.py
-rw-r--r-- 1 myuser mygrp    0 Apr  7 09:23 pip-21.0.1.virtualenv
-rw-r--r-- 1 myuser mygrp    0 Apr  7 09:23 setuptools-52.0.0.virtualenv
-rw-r--r-- 1 myuser mygrp    0 Apr  7 09:23 wheel-0.36.2.virtualenv

They seem to be empty, even the pip folder has zero-length files:

$ ls -al pip
total 8.0K
drwxr-xr-x 13 myuser mygrp 4.0K Apr  7 09:23 _internal
drwxr-xr-x 19 myuser mygrp 4.0K Apr  7 09:23 _vendor
-rw-r--r--  1 myuser mygrp    0 Apr  7 09:23 __init__.py
-rw-r--r--  1 myuser mygrp    0 Apr  7 09:23 __main__.py

Is this intentional?

I did all of this using previous poetry versions (1.1.0 and 1.1.4 if I remember correctly). I see the same behavior.

I see empty site-packages if I use:

  • pyenv installed python
  • python that I compiled myself

site-packages are populated if I use the system python.

@kitmonisit

The rubber ducky problem solving method has been demonstrated. Writing down my problem in #3628 (comment) has helped a lot in framing it.

This is what I did to get around the problem of empty site-packages

# system python is only 3.7. I need 3.8.
$ pyenv install 3.8.8
$ pyenv versions
* system (set by ~/.pyenv/version)
  3.8.8
# Use system python to do all the poetry initialization
$ pyenv global system
$ python --version
Python 3.7.3

$ mkdir debug
$ cd debug
$ poetry new somepackage
$ cd somepackage
$ poetry env use ~/.pyenv/versions/3.8.8/python
$ poetry install
$ poetry shell
$ python --version
Python 3.8.8

That should do it.

@ErikBjare

I’ve stumbled into the same issue today, seemingly out of the blue. This morning I did a system upgrade (pacman -Syu) and after the first poetry install the message started appearing.

I had used pyenv to use Python 3.7.8. I tried clearing the cache and recreating the venv, but no luck. I then tried the system Python to create the venv (Python 3.9.2), but same thing.

I could run poetry update repeatedly in a clean environment, but after the first run of poetry install the message appears every time (even on poetry update).

I then tried installing another project, which worked fine. Then I tried installing the old project (https://github.com/ActivityWatch/aw-client) again, which gave the output:

Installing dependencies from lock file

Package operations: 9 installs, 0 updates, 0 removals

  • Installing lazy-object-proxy (1.6.0)
  • Installing wrapt (1.12.1)
  • Installing astroid (2.5.3)
  • Installing coverage (5.5)
  • Installing isort (5.8.0)
  • Installing mccabe (0.6.1)
  • Installing pylint (2.7.4)
  • Installing pytest-cov (2.11.1)
  • Installing tabulate (0.8.9)

Installing the current project: aw-client (0.3.1)

I assume one of these dependencies are to blame, but I don’t know which (and yes, they are all up to date).

Edit: Weirdly enough the same issue doesn’t seem to occur in CI (ActivityWatch/aw-client#56), so perhaps one of the dependencies aren’t to blame after all? Or it’s just that poetry install or similar is never ran twice…


Edit: The issue ended up being the .egg-link directories that were in various places in my project. I deleted them, and now no issues!

Found them all with: find . -maxdepth 2 | grep .egg-info

@brylie

I’m not sure if this is useful, but I was having this issue when deploying via Docker to Digital Ocean App Platform. I changed my Docker image base as such:

from:

FROM python:3.8.1-slim-buster

to:

@dbalabka

In my case poetry fails with a similar error when I’m trying to add a package:

poetry add "git+https://github.com/streamlit/streamlit.git#0.81.2.dev20210506"
loganasherjones, mrkn, Casyfill, PaulBenn-UnlikelyAI, serramatutu, grugna, stromilas, magomar, martyn-smith, dmwyatt, and 5 more reacted with thumbs up emoji

@debonzi-geru

I am not sure if it has anything to do with what you guys are experiencing but after get this problem localy in serveral projects and find this issue I started to debug poetry and found out that the issue was happening because, for some reason, I had a .egg-info directory. I could than see that poetry was checking on it for installed packages and there was a distribution object with no name in its metadata.

name = canonicalize_name(distribution.metadata[«name»])

Reproducing the error: (I dont know how it would behave with a valid .egg-info directory)

# Working as expected
~/open-offertator$ poetry install
Installing dependencies from lock file

No dependencies to install or update

Installing the current project: open_offertator (0.1.0)

# Create a .egg-info directory
~/open-offertator$ mkdir open_offertator.egg-info/

# Try poetry again
~/open-offertator$ poetry install

  TypeError

  expected string or bytes-like object

  at ~/.pyenv/versions/3.9.4/lib/python3.9/site-packages/poetry/core/utils/helpers.py:27 in canonicalize_name
       23│ _canonicalize_regex = re.compile(r"[-_]+")
       24│ 
       25│ 
       26│ def canonicalize_name(name):  # type: (str) -> str
    →  27│     return _canonicalize_regex.sub("-", name).lower()
       28│ 
       29│ 
       30│ def module_name(name):  # type: (str) -> str
       31│     return canonicalize_name(name).replace(".", "_").replace("-", "_")

I hope it helps.

brechtm, jevinskie, ErikBjare, rbarden3, jaepil-choi, stromilas, soyantonio-w, ybryan, jeff-hykin, LukeLangefelsTR, and 2 more reacted with heart emoji

@rpocase

I have a minimal reproduction pyproject.toml file that may be the same issue. I was trying to add a spacy model directly to my deps (the en-core-web-sm line below), but I ultimately had to specify it as a URL dep to get poetry.lock to generate.

[tool.poetry]
name = "failure-to-install"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.9"
en-core-web-sm = { git = "https://github.com/explosion/spacy-models", tag = "en_core_web_sm-3.0.0" }

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Hope this helps!

@jdkern11

This has nothing to do with anaconda, I’ve had this without using anaconda.

#3628 (comment)

For some of us, it is directly due to anaconda.

@jdkern11

I am currently facing this without anaconda (actually deactivated an anaconda env beacuse of some comments above) on pure venv. Still no recourse.
poetry install works fine the first time,
but the second time around, fails with this errorm which is really bizarre.

Did you try reinstalling poetry, making sure your anaconda environment wasn’t active during install? I’d use the same python binary whenever you run poetry commands.

@MaikuMori

For some of us, it is directly due to anaconda.

It looks like it triggers the bug, but there are other triggers too.

For some reason, the download is corrupted, which then corrupts the cache. Without clearing the cache, it will continue to install the corrupted version.

And by corrupted I mean the METADATA file being empty for the package.

@Matesanz

For me it was related to the .venv folder created by poetry:

👉 Deleting that folder and running poetry update worked for me 😄 .

@ekalosak

I’m using pyenv and poetry with the pyenv-virtualenv extension.
Along the lines of @Matesanz ‘s solution,

pyenv virtualenv-delete <offending-venv>
pyenv virtualenv <python-version> <offending-venv>
poetry install

worked for me.

jeff-hykin

added a commit
to jeff-hykin/poetry
that referenced
this issue

Mar 4, 2022

@jeff-hykin

@LSaldyt

What worked for me:
poetry env list --full-path which returns path
rm -rf path
poetry update
For me, the issue was triggered by installing from a git URL, which worked initially and then became corrupted. I was only using poetry, not conda and only using pip implicitly.

@aalok-sathe

Even after deleting the .venv, poetry install or poetry update continues to give me this

  TypeError

  expected string or bytes-like object

@marj3220

Try deleting the poetry.lock also before the poetry install and update

@ElricleNecro

I’m not sure why you’d use poetry with anaconda? Poetry is an environment manager for virtual envs, why would you make a conda environment too? In my case, I had previously used anaconda environments and use them for some old projects still, while now having migrated solely to poetry. I just needed to switch out of that old anaconda env when switching to a separate poetry managed project.

Because we have as deps some software available only in conda and not pypi (and they won’t be), but we can still use a big part without the need for a conda environment.
Plus, were I work, it is used to handle all aspect of the development (like publishing the package to a private pypi, handling the dependencies, etc), not just the environment (on which it can causes us some trouble when we have to work with conda).

@aalok-sathe You are running poetry inside a conda environment ? If yes, you can try to run the following snippet:

from os import environ, remove, rmdir
from shutil import rmtree
from sys import version_info

import importlib_metadata as metadata

MODULES_PATH = [
    f"{environ['CONDA_PREFIX']}/lib/python{version_info.major}.{version_info.minor}/site-packages",
    f"{environ['HOME']}/.local/lib/python{version_info.major}.{version_info.minor}/site-packages",
]

if __name__ == '__main__':
    for pkg in filter(lambda a: a.metadata["name"] is None, metadata.distributions(path=MODULES_PATH)):
        print("Erasing", pkg._path)
        if pkg._path.is_dir():
            rmtree(pkg._path)
        else:
            remove(pkg._path)

(I’ve already posted it, but can’t recall in which issue).

This was referenced

May 19, 2022

@orionlee

The TypeError «expected string or bytes-like object» appears to obscure various kinds of issues people posted here. It’d be great if the actual underlying root causes can be surfaced instead.

In my case, I ran into the issue in the following combination:

  1. create and use a conda env
  2. use poetry install to install package A from source.
  3. use poetry install to install package B from source.

TypeError is encountered in installing package B.

I later realize doing poetry install on multiple packages in a single conda environment is probably problematic, if not outright unsupported. If that’s indeed the case, it’d be great if a more explicit error is raised instead.

@abn
abn

mentioned this issue

Jun 6, 2022

@adam-grant-hendry

Still getting this error on Poetry 1.1.13. I have no .egg-info folders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.

@neersighted

Still getting this error on Poetry 1.1.13. I have no .egg-info folders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.

The fix is in Poetry 1.2b2 — Poetry 1.1.x is not seeing releases as we work up to a full release of 1.2.

@adam-grant-hendry

Still getting this error on Poetry 1.1.13. I have no .egg-info folders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.

The fix is in Poetry 1.2b2 — Poetry 1.1.x is not seeing releases as we work up to a full release of 1.2.

Yup. Discovered that after commenting about your poor documentation.

For me, version 1.2.0b2 found multiple kiwisolver .dist-info folders in my site-packages. One for an older version and one for the latest. I had no .egg-info folders anywhere, so I think this was the culprit.

@neersighted

Still getting this error on Poetry 1.1.13. I have no .egg-info folders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.

The fix is in Poetry 1.2b2 — Poetry 1.1.x is not seeing releases as we work up to a full release of 1.2.

Yup. Discovered that after commenting about your poor documentation.

Poetry is free software developed by volunteers without any expectation of warranty. I am sorry that you had to struggle to find the relevant documentation, and I will make a note of improving it before release. Note that we are still in beta for 1.2 and things are therefore still in flux. Please refrain from using an unnecessarily hostile or confrontational tone.

For me, version 1.2.0b2 found multiple kiwisolver .dist-info folders in my site-packages. One for an older version and one for the latest. I had no .egg-info folders anywhere, so I think this was the culprit.

I’m glad you were able to sort the issues out. Please test 1.2 in your environment and open issues for any regressions or deficiencies (like documentation) you may find.

@adam-grant-hendry

Apologies, my frustration came out and that was unprofessional. I appreciate your help. Thank you for responding so quickly.

@netsettler

I tried using 1.2.0rc1 and it fails a different way so I can’t tell if it fixes the name problem. It’s in a similar area of code and has the same kind of feel, so I figured I’d at least mention it here since I really don’t have time to open a whole new ticket:

  • Updating certifi (2021.10.8 /Users/kentpitman/Library/Caches/pypoetry/artifacts/61/99/d0/9b77a70db9834d3d8281903159311224b92e6ac1cdab98dca7f6367740/certifi-2021.10.8-py2.py3-none-any.whl -> 2022.6.15): Failed

  AssertionError

  

  at fsenv/lib/python3.7/site-packages/poetry/utils/env.py:343 in find_distribution_files_with_name
       339│     ) -> Iterable[Path]:
       340│         for distribution in self.distributions(
       341│             name=distribution_name, writable_only=writable_only
       342│         ):
    →  343│             assert distribution.files is not None
       344│             for file in distribution.files:
       345│                 if file.name == name:
       346│                     yield Path(
       347│                         distribution.locate_file(file),  # type: ignore[no-untyped-call]

That assertion does behave helpfully because it stops everything dead in its tracks. Is it really that fatal? I would think it would be less disruptive to merge lines 343 and 344 as

for file in distribution.files or []:  # sometimes distribution.files is None

possibly adding a log.warning if you want someone to know about the problem? Putting assert in production code seems iffy. I thought that was only for testing.

@dimbleby

if you care then raise a new ticket and say how to reproduce, no point in making updates in unrelated and already-closed issues

@neersighted

Definitely do open a new issue if you can reproduce this consistently in a container/VM.

With regard to assert in production code, it’s there for typing purposes and is very helpful for catching this kind of bug. It should be impossible for a None to sneak in there according to the intention of the authors of that section of code, and yet it happened — this is why typing (and typing asserts) are very helpful for writing robust Python code.

@netsettler

I’ll try to open a ticket if I get time, but as you probably know, it takes time to do a good ticket that isn’t overly complicated. It helps to know that you think this is not a known problem, since that makes the investment in time seem more justified. Thanks.

@aalok-sathe

@neersighted

Still experience this :(

This is a closed issue that could refer to a wild assortment of different root causes — if you are running into this, please narrow it down to a minimal reproduction and open an issue so we can try to understand how to solve it.

@python-poetry
python-poetry

locked as off-topic and limited conversation to collaborators

Sep 2, 2022

Labels

kind/bug

Something isn’t working as expected

The following error occurs when parsing web page data with beautifulSoup and processing data with regular expressions:
TypeError: Expected string or bytes-like object TypeError: Expected string or bytes-like object TypeError: Expected string or bytes-like object
It is generally caused by data type mismatch.
There are six standard data types in Python3: 

Print (type(object)) to check the current data type, where object is the object to query.

First, there is a code that looks like this:

import re
import requests
from bs4 import BeautifulSoup
import lxml

#get the html data
urlSave = "https://www.douban.com/people/yekingyan/statuses"
req = requests.get(urlSave)
soup = BeautifulSoup(req.text,'lxml')

# After parsing beautifulsoup, get the required data
times = soup.select('div.actions > span')
says = soup.select('div.status-saying > blockquote')

And then I’m going to look at it and I’m going to get the data what is the numeric type

print('says:',type(says))

The result: Says: lt; class ‘list’>
This tells us that the data selected from beautifulSoup in soup.select() is of the list type.
Next, extract the data in the list separately

#Traversing the output
for say in says:
    print(type(say))

Let’s see what type it is
The result: <<; class ‘bs4.element.Tag’> , different from the above six types
Beautiful Soup converts a complex HTML document into a complex tree structure, where each node is a Python object. All objects can be classified into four types:
TagNavigableStringBeautifulSoupComment
Use regular expressions directly to the data

for say in says:
    # Regular expressions to get the necessary data
    say = re.search('<p>(.*?)</p>',say)

There is an error
TypeError: expected string or bytes-like object
Therefore, before the regular expression, the problem is solved by converting the data type. As follows:

for say in says:
    # Convert the data type, otherwise an error will be reported
    say = str(say)
    # Regular expressions to get the necessary data
    say = re.search('<p>(.*?)</p>',say)

Read More:

  • Python PIP TypeError: expected str, bytes or os.PathLike object, not int
  • [Solved] python-sutime Error: the JSON object must be str, bytes or bytearray, not ‘java.lang.String‘
  • [Solved] TypeError: Object of type ‘bytes’ is not JSON serializable
  • [Solved] pycocotools Install Error: ERROR: Error expected str, bytes or os.PathLike object, not NoneType while executing
  • Python TypeError: coercing to Unicode: need string or buffer, NoneType found
  • [Solved] Python3.9 Pycropto RSA Error: TypeError: can’t concat str to bytes
  • [Solved] Python Error: TypeError: write() argument must be str, not bytes
  • Python TypeError: not all arguments converted during string formatting [Solved]
  • [Solved] error: when using the property decorator in Python, an error occurs: typeerror: descriptor ‘setter’ requires a ‘property’ object but
  • [How to Solve] Python TypeError: ‘int‘ object is not subscriptable
  • Python scatter chart error: TypeError: object of type ‘NoneType’ has no len()
  • [Solved] RuntimeError: unexpected EOF, expected 73963 more bytes. The file might be corrupted.
  • [Solved] TypeError: not all arguments converted during string formatting
  • Python Error: SyntaxError: (unicode error) ‘unicodeescape‘ codec can‘t decode bytes in position 2-3:
  • TypeError: Decimal type object is not a JSON serialization solution
  • Django Issues: TypeError: “Settings” object is irreversible
  • Python Pandas Typeerror: invalid type comparison
  • [Solved] Python Error: IndentationError: expected an indented block
  • How to Solve Python AttributeError: ‘dict’ object has no attribute ‘item’
  • Tips for Python 3 string.punctuation

Уведомления

  • Начало
  • » Python для новичков
  • » Множественные замены в текстовом файле

#1 Авг. 25, 2017 11:55:14

Множественные замены в текстовом файле

Добрый день.
Подскажите пожалуйста как в текстовом файле сделать замену по шаблону, с сохранением строк и последующей записью.

Например. Нужно заменить во всем файле “age” на “years” и “fname” на “surname”

 pname 1; fname 1; age 1
pname 2; fname 2; age 2
pname 3; fname 3; age 3

Офлайн

  • Пожаловаться

#2 Авг. 25, 2017 12:10:00

Множественные замены в текстовом файле

Метод replace

С дураками и сектантами не спорю, истину не ищу.
Ели кому-то правда не нравится, то заранее извиняюсь.

Офлайн

  • Пожаловаться

#3 Авг. 25, 2017 12:24:43

Множественные замены в текстовом файле

Почему выдает ошибку?

File “C:Python 3.6libre.py”, line 191, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

 def parsing():
    rpl = [['Нормальный ', ''],
           ['Дверь', ''],
           ['Дата: ', ''],
           [' Область: ', '']]
    f_in = open('in.csv', 'r')
    f_out = open('out.txt', 'w', encoding='utf-8')
    for string in rpl:
        input_file = re.sub(str(string[0]),str(string[1]),f_in)  
    f_out.write(input_file)
    f_in.close()
    f_out.close()

Отредактировано Ubhra (Авг. 25, 2017 12:57:48)

Офлайн

  • Пожаловаться

#4 Авг. 25, 2017 13:12:05

Множественные замены в текстовом файле

Ubhra
Почему выдает ошибку?

в re.sub третий параметр должен быть строкой или набором байт. а у тебя это объект файла.

Офлайн

  • Пожаловаться

#5 Авг. 25, 2017 13:43:41

Множественные замены в текстовом файле

open(‘in.csv’, ‘r’)
должен ведь возвращать содержимое файла одной строкой

П.С. Аааа точно его же прочесть нужно:

 input_file = re.sub(str(string[0]),str(string[1]),f_in.read())

Вот только он создает пустой файл.

Отредактировано Ubhra (Авг. 25, 2017 13:46:35)

Офлайн

  • Пожаловаться

#6 Авг. 25, 2017 14:05:51

Множественные замены в текстовом файле

Вот так частично заработало, но изменяет только первую запись:

 def parsing():
    rpl = [['Нормальный ', ''],['Дверь', ''],['Дата: ', ''],[' Область: ', '']]
    f_in = open('in.csv', 'r')
    f_out = open('out.txt', 'w', encoding='utf-8')
    for string in rpl:
        f_out.write(re.sub(str(string[0]),str(string[1]),f_in.read()))
    f_in.close()
    f_out.close()
    print('ok')

Офлайн

  • Пожаловаться

#7 Авг. 25, 2017 14:29:02

Множественные замены в текстовом файле

Питон зверюга всеядная. У меня так заработало :D :

 def parsing():
    f_in = open('in.csv', 'r')
    f_out = open('out.txt', 'w', encoding='utf-8')
    nf1 = re.sub(str('Нормальный вход по ключу'),str('вход'),f_in.read())
    nf2 = re.sub(str('Нормальный выход по ключу'),str('выход'),nf1)
    nf3 = re.sub(str('Дверь'),str(''),nf2)
    nf4 = re.sub(str('Дата: '),str(''),nf3)
    nf5 = re.sub(str(' Область: '),str(''),nf4)
    nf6 = re.sub(str('"'),str(''),nf5)
    f_out.write(nf6)
    f_in.close()
    f_out.close()
    print('ok')

Офлайн

  • Пожаловаться

#9 Апрель 25, 2020 21:17:13

Множественные замены в текстовом файле

Добрый день!
Не подскажете, почему выдаёт ту же самую “зловредную” ошибку “TypeError: expected string or bytes-like object”?

import re, urllib.request
class HTMLFinder:
def findTag(website, tag):
url = urllib.request.urlopen(website)
tags = url.read()
tagsF = ‘’’»’
for line in tags:
if re.search(tag, line):
tagsF = tagsF + line + ‘/n’
if len(tagsF) < 3: return False
else: return tagsF

Ну я в принципе-то всё правильно написал, пробелы есть, а пишет всё то же(((

Отредактировано PyBeg (Апрель 25, 2020 21:19:47)

Офлайн

  • Пожаловаться

#10 Апрель 26, 2020 04:21:49

Множественные замены в текстовом файле

PyBeg
Не подскажете, почему выдаёт ту же самую “зловредную” ошибку “TypeError: expected string or bytes-like object”?

Потому что принятые данные в байтах надо раскодировать через метод .decode() в юникод-строку. Для раскодирования надо использовать кодировку, которая передаётся вместе с данными. Можно и наугад раскодировать, использовав распространённую кодировку utf-8. Для одного случая подойдёт. Для общего же случая (когда сайт произвольный) надо узнавать кодировку, прежде чем раскодировать, и для этого нужно дополнительный код писать.

Отредактировано py.user.next (Апрель 26, 2020 04:22:32)

Офлайн

  • Пожаловаться

  • Начало
  • » Python для новичков
  • » Множественные замены в текстовом файле

Понравилась статья? Поделить с друзьями:
  • Expected function or variable vba ошибка
  • Expected error junit
  • Expected error assetto corsa
  • Expected end sub ошибка vba
  • Expected calibration error