python os.walk displays mixed windows and unix paths

I am trying to identify all files with certain names in a folder. I am using standard code to do that looking like this:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        print(os.path.join(paths, file))

My problem is about the output of this code in windows machine, basically dynamic parts of the path have wrong slash sign:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesAesSheetNumberEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedDetailEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentTypeEntity.java

start folder which was given is:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aim

and the folder separator is unix one: "/"

while all subsequent subfolders found by os.walk function have windows slash instead: ""

So at the end I have invalid path which cannot be used straight away. Is this a bug in python os library or what actually?

Currently I can easily replace wrong separator with the right one but I am wondering if it is the only way?

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

asked Nov 26 '18 at 12:37

smoczyna

112110

I thought the Python standard was os.sep.join(sequence_to_join) ? I would guess that the use of os.path here may be what's causing your issue

– Andrew
Nov 26 '18 at 12:40

1

The paths are not actually wrong, they will work with the mix of slashes. Both are valid. Windows has two path separators and it doesn't matter if you mix them.

– Martijn Pieters♦
Nov 26 '18 at 12:41

If you want to normalise the paths to a single separator, do so explicitly, using os.path.normpath()

– Martijn Pieters♦
Nov 26 '18 at 12:42

Your example seems mixed up: path starts with D:JAJA (no slash), is that really what you get?

– Joël
Nov 26 '18 at 12:43

1

@Joël: lets assume that that's a simple 'anonymise the string for Stack Overflow posting' editing error. I added the slashes back in.

– Martijn Pieters♦
Nov 26 '18 at 12:47

|
show 2 more comments

I am trying to identify all files with certain names in a folder. I am using standard code to do that looking like this:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        print(os.path.join(paths, file))

My problem is about the output of this code in windows machine, basically dynamic parts of the path have wrong slash sign:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesAesSheetNumberEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedDetailEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentTypeEntity.java

start folder which was given is:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aim

and the folder separator is unix one: "/"

while all subsequent subfolders found by os.walk function have windows slash instead: ""

So at the end I have invalid path which cannot be used straight away. Is this a bug in python os library or what actually?

Currently I can easily replace wrong separator with the right one but I am wondering if it is the only way?

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

asked Nov 26 '18 at 12:37

smoczyna

112110

I thought the Python standard was os.sep.join(sequence_to_join) ? I would guess that the use of os.path here may be what's causing your issue

– Andrew
Nov 26 '18 at 12:40

1

The paths are not actually wrong, they will work with the mix of slashes. Both are valid. Windows has two path separators and it doesn't matter if you mix them.

– Martijn Pieters♦
Nov 26 '18 at 12:41

If you want to normalise the paths to a single separator, do so explicitly, using os.path.normpath()

– Martijn Pieters♦
Nov 26 '18 at 12:42

Your example seems mixed up: path starts with D:JAJA (no slash), is that really what you get?

– Joël
Nov 26 '18 at 12:43

1

@Joël: lets assume that that's a simple 'anonymise the string for Stack Overflow posting' editing error. I added the slashes back in.

– Martijn Pieters♦
Nov 26 '18 at 12:47

|
show 2 more comments

I am trying to identify all files with certain names in a folder. I am using standard code to do that looking like this:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        print(os.path.join(paths, file))

My problem is about the output of this code in windows machine, basically dynamic parts of the path have wrong slash sign:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesAesSheetNumberEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedDetailEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentTypeEntity.java

start folder which was given is:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aim

and the folder separator is unix one: "/"

while all subsequent subfolders found by os.walk function have windows slash instead: ""

So at the end I have invalid path which cannot be used straight away. Is this a bug in python os library or what actually?

Currently I can easily replace wrong separator with the right one but I am wondering if it is the only way?

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

asked Nov 26 '18 at 12:37

smoczyna

112110

I am trying to identify all files with certain names in a folder. I am using standard code to do that looking like this:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        print(os.path.join(paths, file))

My problem is about the output of this code in windows machine, basically dynamic parts of the path have wrong slash sign:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesAesSheetNumberEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedDetailEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentReceivedEntity.java

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aimaesDocumentTypeEntity.java

start folder which was given is:

D:/JAJA/Projects/DAF/AIM/WEBAPP/trunk/src/main/java/ie/gov/agriculture/aim

and the folder separator is unix one: "/"

while all subsequent subfolders found by os.walk function have windows slash instead: ""

So at the end I have invalid path which cannot be used straight away. Is this a bug in python os library or what actually?

Currently I can easily replace wrong separator with the right one but I am wondering if it is the only way?

python windows python-os path-separator

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

asked Nov 26 '18 at 12:37

smoczyna

112110

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

asked Nov 26 '18 at 12:37

smoczyna

112110

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

edited Nov 26 '18 at 12:46

Martijn Pieters♦

723k14125392343

asked Nov 26 '18 at 12:37

smoczyna

112110

asked Nov 26 '18 at 12:37

smoczyna

112110

asked Nov 26 '18 at 12:37

smoczyna

112110

I thought the Python standard was os.sep.join(sequence_to_join) ? I would guess that the use of os.path here may be what's causing your issue

– Andrew
Nov 26 '18 at 12:40

1

The paths are not actually wrong, they will work with the mix of slashes. Both are valid. Windows has two path separators and it doesn't matter if you mix them.

– Martijn Pieters♦
Nov 26 '18 at 12:41

If you want to normalise the paths to a single separator, do so explicitly, using os.path.normpath()

– Martijn Pieters♦
Nov 26 '18 at 12:42

Your example seems mixed up: path starts with D:JAJA (no slash), is that really what you get?

– Joël
Nov 26 '18 at 12:43

1

@Joël: lets assume that that's a simple 'anonymise the string for Stack Overflow posting' editing error. I added the slashes back in.

– Martijn Pieters♦
Nov 26 '18 at 12:47

|
show 2 more comments

I thought the Python standard was os.sep.join(sequence_to_join) ? I would guess that the use of os.path here may be what's causing your issue

– Andrew
Nov 26 '18 at 12:40

1

The paths are not actually wrong, they will work with the mix of slashes. Both are valid. Windows has two path separators and it doesn't matter if you mix them.

– Martijn Pieters♦
Nov 26 '18 at 12:41

If you want to normalise the paths to a single separator, do so explicitly, using os.path.normpath()

– Martijn Pieters♦
Nov 26 '18 at 12:42

Your example seems mixed up: path starts with D:JAJA (no slash), is that really what you get?

– Joël
Nov 26 '18 at 12:43

1

@Joël: lets assume that that's a simple 'anonymise the string for Stack Overflow posting' editing error. I added the slashes back in.

– Martijn Pieters♦
Nov 26 '18 at 12:47

I thought the Python standard was os.sep.join(sequence_to_join) ? I would guess that the use of os.path here may be what's causing your issue

– Andrew
Nov 26 '18 at 12:40

The paths are not actually wrong, they will work with the mix of slashes. Both are valid. Windows has two path separators and it doesn't matter if you mix them.

– Martijn Pieters♦
Nov 26 '18 at 12:41

If you want to normalise the paths to a single separator, do so explicitly, using os.path.normpath()

– Martijn Pieters♦
Nov 26 '18 at 12:42

Your example seems mixed up: path starts with D:JAJA (no slash), is that really what you get?

– Joël
Nov 26 '18 at 12:43

@Joël: lets assume that that's a simple 'anonymise the string for Stack Overflow posting' editing error. I added the slashes back in.

– Martijn Pieters♦
Nov 26 '18 at 12:47

|
show 2 more comments

1 Answer
1

active

oldest

votes

There is no actual problem here. Windows supports two path separators; the forward and backward slashes are both valid and supported, even when mixed. One is the os.sep (), and the other the os.altsep character (/).

os.path.join() user os.sep to join paths, but won't replace os.altsep in the input paths. os.walk() just uses os.path.join() to build the first element of each (path, files, directories) tuple it generates

If this bothers you, normalise your paths, using the os.path.normpath() function:

On Windows, it converts forward slashes to backward slashes.

So normalise the path passed to os.walk():

for paths, subdirs, files in os.walk(os.path.normpath(start_dir), topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        print(full_path)

or normalise the paths generated in the loop:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        normalised = os.path.normpath(full_path)

        print(normalised)

or normalise the input string:

edited Nov 26 '18 at 12:50

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

it did bother me that the path is mixed because when you try to create a new file with such path it will fail, however you answer fills the bill

– smoczyna
Nov 26 '18 at 14:40

@smoczyna: no, creating a new file with a path that mixes forward and backward slashes will not fail, everything else being equal (i.e. if that path with only forward or only backward slashes can be created, then it can be created with mixed slashes too0.

– Martijn Pieters♦
Nov 26 '18 at 14:50

Well I have permission denied error when I try. It is obviously not true as I am operating within my home folder only.

– smoczyna
Nov 26 '18 at 15:18

@smoczyna: Now try again with os.path.normpath(). You'll get the same exception.

– Martijn Pieters♦
Nov 26 '18 at 15:23

You actually right, that was my fault, not the path, thanks

– smoczyna
Nov 26 '18 at 15:58

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53481285%2fpython-os-walk-displays-mixed-windows-and-unix-paths%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

If this bothers you, normalise your paths, using the os.path.normpath() function:

On Windows, it converts forward slashes to backward slashes.

So normalise the path passed to os.walk():

for paths, subdirs, files in os.walk(os.path.normpath(start_dir), topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        print(full_path)

or normalise the paths generated in the loop:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        normalised = os.path.normpath(full_path)

        print(normalised)

or normalise the input string:

edited Nov 26 '18 at 12:50

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

it did bother me that the path is mixed because when you try to create a new file with such path it will fail, however you answer fills the bill

– smoczyna
Nov 26 '18 at 14:40

@smoczyna: no, creating a new file with a path that mixes forward and backward slashes will not fail, everything else being equal (i.e. if that path with only forward or only backward slashes can be created, then it can be created with mixed slashes too0.

– Martijn Pieters♦
Nov 26 '18 at 14:50

Well I have permission denied error when I try. It is obviously not true as I am operating within my home folder only.

– smoczyna
Nov 26 '18 at 15:18

@smoczyna: Now try again with os.path.normpath(). You'll get the same exception.

– Martijn Pieters♦
Nov 26 '18 at 15:23

You actually right, that was my fault, not the path, thanks

– smoczyna
Nov 26 '18 at 15:58

add a comment |

If this bothers you, normalise your paths, using the os.path.normpath() function:

On Windows, it converts forward slashes to backward slashes.

So normalise the path passed to os.walk():

for paths, subdirs, files in os.walk(os.path.normpath(start_dir), topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        print(full_path)

or normalise the paths generated in the loop:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        normalised = os.path.normpath(full_path)

        print(normalised)

or normalise the input string:

edited Nov 26 '18 at 12:50

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

it did bother me that the path is mixed because when you try to create a new file with such path it will fail, however you answer fills the bill

– smoczyna
Nov 26 '18 at 14:40

@smoczyna: no, creating a new file with a path that mixes forward and backward slashes will not fail, everything else being equal (i.e. if that path with only forward or only backward slashes can be created, then it can be created with mixed slashes too0.

– Martijn Pieters♦
Nov 26 '18 at 14:50

Well I have permission denied error when I try. It is obviously not true as I am operating within my home folder only.

– smoczyna
Nov 26 '18 at 15:18

@smoczyna: Now try again with os.path.normpath(). You'll get the same exception.

– Martijn Pieters♦
Nov 26 '18 at 15:23

You actually right, that was my fault, not the path, thanks

– smoczyna
Nov 26 '18 at 15:58

add a comment |

If this bothers you, normalise your paths, using the os.path.normpath() function:

On Windows, it converts forward slashes to backward slashes.

So normalise the path passed to os.walk():

for paths, subdirs, files in os.walk(os.path.normpath(start_dir), topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        print(full_path)

or normalise the paths generated in the loop:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        normalised = os.path.normpath(full_path)

        print(normalised)

or normalise the input string:

edited Nov 26 '18 at 12:50

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

If this bothers you, normalise your paths, using the os.path.normpath() function:

On Windows, it converts forward slashes to backward slashes.

So normalise the path passed to os.walk():

for paths, subdirs, files in os.walk(os.path.normpath(start_dir), topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        print(full_path)

or normalise the paths generated in the loop:

for paths, subdirs, files in os.walk(start_dir, topdown=True):

    for file in files:

        full_path = os.path.join(paths, file)

        normalised = os.path.normpath(full_path)

        print(normalised)

or normalise the input string:

edited Nov 26 '18 at 12:50

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

edited Nov 26 '18 at 12:50

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

answered Nov 26 '18 at 12:44

Martijn Pieters♦

723k14125392343

it did bother me that the path is mixed because when you try to create a new file with such path it will fail, however you answer fills the bill

– smoczyna
Nov 26 '18 at 14:40

@smoczyna: no, creating a new file with a path that mixes forward and backward slashes will not fail, everything else being equal (i.e. if that path with only forward or only backward slashes can be created, then it can be created with mixed slashes too0.

– Martijn Pieters♦
Nov 26 '18 at 14:50

Well I have permission denied error when I try. It is obviously not true as I am operating within my home folder only.

– smoczyna
Nov 26 '18 at 15:18

@smoczyna: Now try again with os.path.normpath(). You'll get the same exception.

– Martijn Pieters♦
Nov 26 '18 at 15:23

You actually right, that was my fault, not the path, thanks

– smoczyna
Nov 26 '18 at 15:58

add a comment |

it did bother me that the path is mixed because when you try to create a new file with such path it will fail, however you answer fills the bill

– smoczyna
Nov 26 '18 at 14:40

@smoczyna: no, creating a new file with a path that mixes forward and backward slashes will not fail, everything else being equal (i.e. if that path with only forward or only backward slashes can be created, then it can be created with mixed slashes too0.

– Martijn Pieters♦
Nov 26 '18 at 14:50

Well I have permission denied error when I try. It is obviously not true as I am operating within my home folder only.

– smoczyna
Nov 26 '18 at 15:18

@smoczyna: Now try again with os.path.normpath(). You'll get the same exception.

– Martijn Pieters♦
Nov 26 '18 at 15:23

You actually right, that was my fault, not the path, thanks

– smoczyna
Nov 26 '18 at 15:58

it did bother me that the path is mixed because when you try to create a new file with such path it will fail, however you answer fills the bill

– smoczyna
Nov 26 '18 at 14:40

@smoczyna: no, creating a new file with a path that mixes forward and backward slashes will not fail, everything else being equal (i.e. if that path with only forward or only backward slashes can be created, then it can be created with mixed slashes too0.

– Martijn Pieters♦
Nov 26 '18 at 14:50

Well I have permission denied error when I try. It is obviously not true as I am operating within my home folder only.

– smoczyna
Nov 26 '18 at 15:18

@smoczyna: Now try again with os.path.normpath(). You'll get the same exception.

– Martijn Pieters♦
Nov 26 '18 at 15:23

You actually right, that was my fault, not the path, thanks

– smoczyna
Nov 26 '18 at 15:58

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Nsryjdtyk