[ISSUE] dbuilts.fs.ls('file:/local-path/') lists only files, not the folders #992
Description
Description
in databricks-sdk==0.56.0 (and possibly others), when running locally on a VM or laptop, the dbuilts.fs.ls'file:/local-path/') lists only files, not the folders. I think its a bug.
Normally, the dirs should be listed, the returned FileInfo object should have name and path of a dir ending with '/'.
Reproduction
dbutils.fs.ls('file:/<your home folder>') it will just print files, and there is plenty other folders that you should see
Expected behavior
it should return directories as well, dirs should end with '/'
Is it a regression?
not sure.
Additional context
By looking into the code behind, it looks like code responsible for recursion omits dirs when recursion flag is off:
def list(self, *, recursive=False) -> Generator[files.FileInfo, None, None]:
if not self.is_dir:
meta = self._api.get_status(self.as_string)
yield files.FileInfo(
path=self.as_string,
is_dir=False,
file_size=meta.file_size,
modification_time=meta.modification_time,
)
return
queue = deque([self])
while queue:
next_path = queue.popleft()
for file in self._api.list(next_path.as_string):
if recursive and file.is_dir:
queue.append(self.child(file.path))
if not recursive or not file.is_dir: # <<<HERE>>>: THIS should yield all entries, just like databricks dbutils does. The returned folder should end with /, for example 'important_stuff/' should be returned if it's a folder (dbutils.fs.ls does it like this)
yield file