FHS extension for user home folders

Posted by: aigarius 7 years, 10 months ago

(21 comments)

Justification.


Currently there is a huge mess of files and folders that start with a "." in any users home folder. There is no structure or policy on how applications should choose file and folder names for data that needs to be stored in users home directory. Additionally there is no established consistency between Gnome, KDE and most other applications. Gnome application have part of their configuration information in gconf folder and other part in a gnome subfolder. KDE applications have a complex structure under .kde/. And most other applications either have one file directly in users home folder or have their own dot-folder there.


Problems.


There are two major problems with current ad-hoc approach:




  1. Namespace pollution of users home space. In any desktop system "ls -al" in the home folder would be a very lengthy affair. Moreeven this namespace is folded - KDE and Gnome apps live in their own subfolders.

  2. Lack of structure in folders makes it much harder to properly understand their purpose. For example, if a user has misconfigured an application and would like to reset it to default settings, it currently could be hard to figure out how to do that. Especially without losing his data in that application.


Proposal.


I propose to implement a policy on where an application can store data and configuration information in users home folder. Such specification would be proposed as an extension for FHS, first implemented in Debian and then promoted for inclusion upstream and in other distributions.


Structure.


I propose a following structure for a users home folder:



  • data/appname/ - each application, that stores user data in application-specific format, will have to store this data into this folder. Only configuration information that is essential to load this data would be permitted into this folder (in hidden dot files). For example, Evolution could store emails and email access information here.

  • .conf/appname/ - each application that stores any configuration information whould have to store all of it in this folder. GConf will have to also store applications configuration info here. If this folder would be deleted, application must revert to default configuration, but still be able to fully access user data in data/appname/ or other, generic folders (below). For example, if .conf/evolution/ is erased, then on next start-up Evoution should still be able to access all email that is stored in data/evolution/ without any configuration wizards.

  • .tmp/appname/ - each application that would need to create temporary files that survive reboot, would put them here. These folders can be safely deleted at any time with no data or configuration loss. For example, Firefox cache information and Nautilus thumbnail cache would have to be here.

  • Desktop/ - the desktop folder, just like it is now

  • Media folders - (Documents/, Photos/, Videos/, Music/, ...) a suggestion of basic document structure for user to use or ignore. Applications dealing with particular media types would default to use files from such folders, but should also cope with working with any other folders on users choice. For example, a screensaver with slideshow function would by default use photos from Photos/, but will have to have a configuration option for user to change that setting.

  • Additionally I would propose having a .mounts/ folder where applications could mount user specific media and other mounts, such as FUSE. For example, FUSE could be used for automatic archive introspection.


Benefits



The main benefit for such system for me is twofold:


  • SBackup can be configured to exclude .tmp/ and .mounts/ by default thus saving significant space in backups.

  • A tool can be made to restore settings and/or data for specific applications in an uniform fashion.




So, any suggestions on the idea and on where this should be best discussed?

Currently unrated


Comments

  • Mark Hymers 7 years, 10 months ago

    Doesn't this: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html

    already address at least the config part of the issue?

    Link / Reply
  • Pharao 7 years, 10 months ago

    great idea!

    but the first step would be to make sure that the distributions doesn't change the default config path - Thunderbird for example: .thunderbird or .mozilla-thunderbird

    Link / Reply
  • Stephen Touset 7 years, 10 months ago

    I like the idea of having a structure for dotfiles, but I don't agree to the same extent that we should be forcing names and structure on people.

    Perhaps have a ~/.data, ~/.conf, ~/.tmp, but I wouldn't agree with a standards-mandated Photos/, Music/, Video/, etc. folder. For instance, think about non-English speakers.

    The big use for those directories would be for GUI apps, and GNOME already uses a Desktop/ and Documents/ directory. Let GNOME worry about adding Videos/, Photos/, et al.

    Link / Reply
  • dom 7 years, 10 months ago

    I half agree with you, Stephen. In managing a Mac environment, I find that basically everyone uses the default folders (Desktop, Documents, Pictures, Music, etc) so it does encourage users to keep their files organized.

    To support non-English environments, this should be configurable like with KDE where the desktop, autostart, and document paths can be reconfigured to something sensible. KDE does only allow those three folders to be reconfigured, but that's a workable model.

    Link / Reply
  • Jakob Kaivo 7 years, 10 months ago

    The XDG spec goes part way on this, with the addition that the directories aren't set in stone, but are set by the environment. This is absolutely crucial. Of course, it's just a matter of changing programs to comply.

    Link / Reply
  • aigarius 7 years, 10 months ago

    XDG spec is interesting as it adds one more point - application data that is not really user data and is not config, for example plugins and extensions.
    However I do not think that it is realistic to add full XDG support to all apps in a sensible time-frame. However simply changing static file names to other static file names could be done for most software before next Debian release.
    Non-dot folders would be only suggestions for users to use or ignore.

    Link / Reply
  • Josef Assad 7 years, 10 months ago

    It certainly would be nice to be able to look in one place for all of one's dotfiles and such.

    The obvious challenge is that there's a plethora of applications out there that expect these things in ~ .

    Though, probably this dotfile centralization could be accomplished pretty easily by having the dotfiles being symbolic links to the actual config files. For example:

    ~/.muttrc would be a link to:
    ~/.allmyrcfiles/.muttrc

    And then it would be some simple scripts before tis whole process would be automated.

    Link / Reply
  • Scott Robinson 7 years, 10 months ago

    I agree with your sentiment, but I think the implementation is flawed. It is extraordinarily convenient for all application associated data to be located in one directory.

    This convenience has ramifications on centralized configuration, ease of deletion, and tracking changes.

    Splitting it all up seems very inconvenient. To be honest, this is part of my problem with gconf.

    Link / Reply
  • Sven Müller 7 years, 10 months ago

    Scott: There are two ways to sort the data two or more applications store:
    1) By use, as Aigarius suggests:
    This makes backup solutions simpler, since you can simply exclude ~/.tmp or ~/.cache or whatever the name would be to exclude any applications temporary files. This would work even for unknown applications. And Backups are just one use case here.
    2) by application as you suggest.
    This makes handling addition and removal of certain applications easier.
    I personally prefer the by-use sorting. But I would be perfectly OK with by-program-sorting if some thinks could be agreed upon, for example that any deletable data (such as the browser caches) should be stored in ~/.apps//tmp or ~/.apps//cache (I prefer the later, since it is analogous to /var/cache) and that the configuration that could be deleted to restore the default configuration is in a directory like ~/.apps//local-config.

    Either way: I generally would support such a proposal/amendment to LSB/FHS. The details would have to be worked out though, including a transition plan for existing users of the affected software.

    Link / Reply
  • Jon 7 years, 10 months ago

    Your comment system just ate my post (and insulted my math ability) and I can't be bothered to write it properly again, so I'll be brief: In short, the best thing about the XDG spec is the support for a user redefining the paths with environment variables. Please ensure to copy this functionality as well if you really do decide on re-inventing the wheel.

    Link / Reply
  • Erich Schubert 7 years, 10 months ago

    Openbox for example already uses ~/.config/openbox; the same apparently applies to last-exit, sonata and xfce.

    One thing I'd really like to see is some separation of data to backup and unimportant data. This goes a bit beyond the .cache / .tmp thing, but for example I consider most configuration unimportant, and I don't care much about themes and similar things I could download. I do however want to backup e.g. my bookmarks and email...

    One thing I'd also put into such a specification is a recommendation set for configuration files. Ideally, all applilcations could parse and write any of these formats, including XML and a nested-INI-file format (i.e. INI-style format with multiple levels). Current configuration file syntax is a MESS, thats why we're using text editors with no assistance to edit these files. It would be nice if we could have a schema description for configuration files, so we could at some point get syntax completion and similar things when editing configuration files. And editors that can verify the syntax to a large extend etc. - so it would perfectly make sense for applications to adopt a common configuration file format.
    I agree that XML isn't the best choice for every application, many applications don't need nesting with mixed content etc. - often a simple key=value format is okay. But we should be able to restrict the number of formats to like 3 or so (e.g. plain key=value, INI-files, XML) and maybe even make a common parsing/writing library for these. That could also abstract config files and gconf and allow users to use either...

    Link / Reply
  • Marco Cabizza 7 years, 10 months ago

    One (secondary) thing I would suggest is to localise the media folders names, i.e. f-spot stores the pictures into a "Photos" directory, while banshee uses "Music" for its library: such can be easily provided translated by default.

    Overall this is a great idea, it's a topic that lacks discussion and documentation.

    Link / Reply
  • Mike 7 years, 10 months ago

    I like you approach to sorting the "dot-files", especially separating the temp/cache files. But I'm worried about the default-folders for photos, videos, music and such. Right now when I look at a "My documents" folder under Windows, almost every application creates a folder for you: My e-books, my builds, my photos, my music... and it is very hard to get rid of these folders because they are re-created without the applications asking you. So please, don't rush and force a structure on users - not everybody wants theses folders.

    Suggestion: Let a wizard (started when the user first starts the desktop environment) ask the user which folders he wants to create or whether he wants to use existing ones, or no predefined folders at all, then store this data for all other applications to use. This will also allow localization of the folder names.

    Link / Reply
  • eknagy 7 years, 10 months ago

    I have a slightly different idea I am using for 3+ years in my applications (see eknagy on sourceforge). There is a data/APP_NAME/ directory where non-binary non-source non-docs non-util files go.
    The recommendation is to have a data/APP_NAME_VERSION (or more likely .apps/APP_NAME_VERSION) with recommendations of the following subdirectory names:
    conf, work (or tmp?), input, skins, templates, ...
    Maybe conf and work should be mandatory and the others are optional.
    It is very handy on long-term.
    So, IceWeasel should look after .apps/mozilla_iceweasel_2_0_0/.config/userprefs.js
    and .apps/mozilla_iceweasel_2_0_0/.work/cache

    Elemer

    Link / Reply
  • FreeSoftNews » Blog Archive » Debian W 7 years, 10 months ago

    [...] http://www.aigarius.com/blog/2007/01/10/fhs-extension-for-user-home-folders/ 6. http://www.pathname.com/fhs/ 7. [...]

    Link / Reply
  • Sander Marechal 7 years, 10 months ago

    There's another thing that needs to be taken into account for this: easily migrating /home from distro to distro. Incidentally there's a discussion about it on Groklaw: http://www.groklaw.net/comment.php?mode=display&sid=20070130091718234&title=Linux+Distros+working+easily+with+each+other&type=article&order=&hideanonymous=0&pid=532990#c532996

    If you're going to define a structure for dotfiles and configuration files inside a user's home, then you could try to address this issue as well somehow. Migrating /home from one distro to another should be as simple as mounting it, without worrying if distro X's dotfiles will mess up distro Y (e.g. incompatible versions)

    Link / Reply
  • Luc Dufresne 7 years, 10 months ago

    About the config part:

    The XDG spec implies to patch every app you use, since very few of them know about $XDG_CONFIG_HOME. So I wrote a LD_PRELOAD-able shared library that intercepts file operations. if a program tries to open a dotfile in $HOME, it is redirected to $XDG_CONFIG_HOME

    http://ordiluc.net/fs/libetc/

    Link / Reply
  • Paul Judd 7 years, 9 months ago

    I agree that application configuration files need to be organized into special directories. I have created a directory own in my home folder that I use for my data files in topics of audio, graphics, maths, text, email, text etc as $HOME was getting too cluttered and when I reinstalled or changed distribution I could keep data. Also as I rearranged partitions and added disk I created extra space in home directory called common for each persons 'own' files readable only to them to separate from reinstalls when application upgrades made data files 'disapear' when structure was re organized. I found many applications created duplicates of data when upgraded but don't know what directory to remove as too busy to delve into it without breaking application.

    Link / Reply
  • Aigarius Blog » Home folder organisation 7 years, 9 months ago

    [...] last post about a FHS amendment to address the structure of user’s home folders, I received a lot of [...]

    Link / Reply
  • Gintautas Miliauskas 7 years, 9 months ago

    It's interesting that you got irritated by the hidden configuration spam too. One of my first kernel hacks was a patch to VFS that transparently translates references to /home/$USER/.foo to /home/$USER/etc/foo. That actually worked, though probably I had left some bugs as the patched kernel wasn't very stable.

    Link / Reply
  • escort girls Dubai 2 years, 5 months ago

    موضوع لا مثيل له ، ومن المثير للاهتمام للغاية بالنسبة لي)))) شكرا www.aigarius.com TEAM

    Link / Reply

New Comment

required
required (not published)
optional

Recent Posts

Archive

2014
2013
2012
2011
2010
2009
2008
2007
2006
2005

Categories

Authors

Feeds

RSS / Atom