Ubuntu annoyance

I downloaded the .torrent for PyWeek #10 today. 400 mb of mad creativity, that’s for sure.

But first stumbling block is an #ubuntu #annoyance. Why oh why doesn’t double-clicking a .py file do the right thing?

The right thing to me is this:

  1. Run “python file.py”
  2. Do not open a terminal window! Many Python games are based on PyGame or Pyglet, both if which open their own window
  3. .. unless something is caught in stdout or stderr; then I want to see what happened

(2) is even more “for sure” if the file ends with “.pyw” which under Windows means exactly “Do not open any console window”.

The thing that could go wrong with this approach, as I see it, is if the Python script, without outputting anything to the stdout/stderr, requests input from stdin first of all. Then, that “algorithm” would fail silently (nothing displayed, a “dead” python interpreter in memory). Maybe some hacker can find a solution to this.

On my Ubuntu 9.04 system, exactly *nothing* happens when I double-click a .py file. When I right-click and choose custom command, and type “python” there, a terminal window flashes by, then closes in a blink, throwing away any error message for me to see (wild guess: wrong working directory?).

/rant from an otherwise positive Ubuntu user/programmer

PS. I really hope 10.04 has fixed this; the way I see it Python is one of the most promising creative multimedia languages for Ubuntu. DS

Advertisements

28 thoughts on “Ubuntu annoyance

  1. I probably misunderstood what you mean, because my objection is this:

    python is a rather general purpose programming language. As such you have no idea (from Gnome’s point of view) what the program will do when executed.

    So it is perfectly plausible that someone will write an app that does the equivalent of “ls”, “grep”, “mv”, “cp” etc, etc, all of which expects arguments or a stream of data.

    Double clicking one of those to execute them would make absolutely no sense. I think the idea of clicking “files” (rather symbols for files in a graphical environment) comes from the habit of using Windows and such, which in many cases has managed to effectively hide from the user the possibility to do stuff via CLI and the power of that 😉

    Please explain what I didn’t get!

  2. The use-case for me, as a python developer and end-user, is downloading Python games from the net.

    Then I unpack-them, probably to the desktop for convenience, with the built-in archiver of Ubuntu.

    After that, I click myself into the new folder on the desktop, and see a “run_game.py” script.

    Then I want to be able to either;

    1. Double-click the run_game.py file to run the game (see behaviour in my blog post above)
    2. Get a Terminal inside that folder (right-click, run terminal “here”)

    Neither of these options exist in a vanilla 9.04 install (my home computer).

    BTW – I fully understand your concerns, and python in general/Gnome in general should not behave like this. But Ubuntu, the linux for human beings, should configure gnome/Nautilus/the-right-technology to behave like this. IMHO. The reason is that Python is more and more used as an application language, rather than a command-line utility language. One use case does not exclude the other, either.

  3. The reason is that Python is more and more used as an application language, rather than a command-line utility language. One use case does not exclude the other, either.

    This is where I beg to differ;-) If you change the default behavior of Gnome (in Ubuntu) when user double-clicks a .py file, you exactly exclude the first use case, with a CLI application.

    Since there is no way for “Gnome/Whatever” to know this, basing things on a feeling that that Python is more and more used as an application language, rather than a command-line utility language seems just wrong 😉

    I think the fault lies in “run_game.py” which doesn’t do what you want. If it’s a GUI game, you probably want to run it as a background process without opening a terminal. If it requires stdin interaction first, you probably want to open it in a terminal.

    I really fail to see the problem here, so please help me out 😉

  4. This is where I beg to differ;-) If you change the default behavior of Gnome (in Ubuntu) when user double-clicks a .py file, you exactly exclude the first use case, with a CLI application.

    Why does configuring the behaviour of Nautilus under Gnome+Ubuntu this way, exclude the CLI use case? I just can’t see that.

    I think the fault lies in “run_game.py” which doesn’t do what you want.

    It doesn’t do _anything_ right now, because .py files have no meaning under Nautilus+Gnome+Ubuntu. I think they should have a meaning.

    One of the major points of Python is platform independence – a program starter/script or similar for *nix:es would be like adding a .bat file for windows, to the archive, thus invalidating the platform independence.

  5. Then we have different definitions for “platform independence” which explains a lot of the differnce in oppinion.

    Here’s my definition 😉

    Python, like for instance Java, is platform independent because it needs an interpreter (in Java’s case a virtual machine). It has nothing to do with how a particular instance of a python script is executed or started – the platform independence lies in the premise that there will be an interpreter for this file on your system.

    Java is able to run wherever there’s a virtural machine. That has nothing to do with whether class files can be executed by double-clicking or not 😉 (In windows, I believe you can run a Java class with a main() method by double-clicking its class file in explorer – a thing I regard as a peculiarity rather than usability feature).

    Likewise, a python script can be interpreted by the python binary, and the existence of this on many platforms is what makes it platform independent – not that you can start the execution/interpretation of the file by exactly the same user action.

    I run 8.04 so I can’t say what would happen in 9.04, but what if you change the perms of the run_game.py to be exec by you? Does it change anything? And what does the shebang in run_game.py tell you? Does it make sense?

    If the file is not executable, I’m not sure double-clicking it should do anything (perhaps open it in some GUI editor?).

    So in short, we have different opinions because we have different expectations for the term “platform independent”. Then we have different expectations for what would happen if we double-click a file symbol in Gnome/whatever wm.

    Windows, I think, have solved this “problem” (I don’t see the problem, but that’s me) by introducing alternate versions of interpreters that can run without opening a terminal, like perlw and pythonw which are interpreters with a different subsystem than cmd.exe (they use explorer instead if I understand correctly).

    Perhaps you can hack together some equivalent for gnome (is there one available already perhaps?) and change the default action for .py for your user to use that?

    I’m sure Gnome will let you choose default action for different filetypes when double-clicked 🙂

    This is an interesting discussion I must say 🙂

  6. Actually I’m not sure any more what the default behaviour of 9.04 is since I’ve tried to add “python” “xterm -e python” and similar by right-clicking .py icons in Nautilus too much. I can tell you it did not perform the steps in the above blog post at least (otherwise I would not have tried, in vain, to add the appropriate action).

    Again, I think you would understand my situation better if you downloaded a couple of source-distribution python games (like the PyWeek #10 bundle at http://pyweek.org for example) and tried running them from Nautilus.

  7. Yes, that worked. In these steps:

    1. Go to game folder in Nautilus.
    2. Copy the path (in location field at top)
    3. Open up a Terminal from Program menu
    4. Write “cd ” + paste path
    5. Type “chmod u+x run_game.py”
    6. Type “./run_game.py”

    Compare this with:

    1. Go to game folder in Nautilus.
    2. Double-click run_game.py

    Now think more than 40 such games. I could write a recursive python script that does the chmod for each run_game.py, because I’m a programmer. It’d probably take me 30 minutes and there’s a chance I’d screw up something.

    What about non-programmers?

  8. Yes, it did 😉 So modify the algorithm to this:

    1. Go to game folder in Nautilus.
    2. Copy the path (in location field at top)
    3. Open up a Terminal from Program menu
    4. Write “cd ” + paste path
    5. Type “chmod u+x run_game.py”
    6. Switch back to Nautilus window
    7. Double-click script

    (I’ve gone through this with 4-5 games now.)

  9. An improved algorithm discovered:

    1. Go to game folder in Nautilus.
    2. Right-click run_game.py, Properties->Rights
    3. Check “Executable” checkbox, close window
    4. Double-click run_game.py

  10. “Now think more than 40 such games. I could write a recursive python script that does the chmod for each run_game.py, because I’m a programmer. It’d probably take me 30 minutes and there’s a chance I’d screw up something.”

    I still think that’s not a Ubuntu problem, but rather a distributor’s problem!

    And you could use:

    find . -name run_game.py -exec chmod u+x {} \;

    😀

  11. To me, it seems more like a windows problem, since it’s probably windows that got people to believe that clicking means running, regardless of executable flags, interpreters and whatnot 😛

    😉

    PS
    Couldn’t reply to your reply, there was no “reply” link on this level.

  12. PS
    Couldn’t reply to your reply, there was no “reply” link on this level.

    I’ll try to turn off the multi-level replies, it just does not work satisfactorily.

    To me, it seems more like a windows problem, since it’s probably windows that got people to believe that clicking means running, regardless of executable flags, interpreters and whatnot 😛

    If you click the firefox icon in the top-level of ubuntu/gnome – does it not run firefox?

    If you click the firefox icon on the Program menu, does it not run firefox?

    If you browse to the firefox binary using Nautilus, does double-clicking not run firefox?

    If I download a binary distribution of a C/SDL-game, and double-click the Game-binary, does it not run the game?

    If I download a source distribution of a Python game, should not the game.py run the game?

    I think it is a quite safe assumption that users expect double-clicking program symbols to mean execute that program.

    Anyway I don’t thing we’re getting anywhere with this discussion. From a usability standpoint, I cannot see how I’m wrong. Especially in a beginner linux distribution, like Ubuntu claims to be.

    It’s a whole other thing on, say, ArchLinux.

  13. A pyhon script source file is not a binary. Does that answer your question? 😉

    And actually, when you run firefox by clicking the icon, you really run a script that in turn runs the executable. And that script has its exec flag set, so you can’t compare to a textfile with python commands in it, without the exec flag set.

    I still think Windows is to blame for this confusion. IIRC Windows used to let the file suffix decide what to do with files when executed. Can you come up with any problems with that solution? 😉

  14. A pyhon script source file is not a binary. Does that answer your question? 😉

    Are you saying all command line utilities & applications on Ubuntu are native binaries? Wasn’t one of your arguments against me grep-written-in-python?

  15. No, you are not listening 😉

    If you download a python source file script, it is not executable by default. Therefore you cannot compare it to the executable scripts and binaries already installed on your system, and therefore nor can you expect the same behavior when double-clicking them!

    Did you check out piinstaller?

    I’m arguing that this is not a Ubuntu problem at all. If you think setting the exec flag of your python text files is awkward, then you should tell that to the distributor. An install script could either set the executable flag for you, or compile it to native executable binary. For ubuntu, the normal distribution is to include applications in a repository so that the user can install them using the application manager or aptitude system.

    Other files, to be installed at own risk, are distributed as tarballs where you have to perform more work in order to install them like sometimes compiling, and other times just configuring and running a simple install script (not by double-clicking it though).

    You were comparing the behaviour to already installed files, which are made executable at install time. You can’t make a fair comparison to a textfile downloaded off of the internet 😉

    Or do you wish that all files downloaded should be executable per default? Or that Ubuntu should use the file name in order to determin that the file is executable or not? 😉

    See?

    Some might argue that it’s very convenient if you just can download a – say – .vbs file using – say – email and when clicking the file, just becuase its name, stuff starts to happen with your computer.

    I don’t think that’s really the preferred behavior.

    Are we getting closer to eachothers views here now?

  16. If you download a python source file script, it is not executable by default. Therefore you cannot compare it to the executable scripts and binaries already installed on your system, and therefore nor can you expect the same behavior when double-clicking them!

    Python applications do not need to be executable; the Python interpreter just need to be able to read them:

    $ python some_file.py

    This is also the most popular way of running python games.

    Did you check out piinstaller?
    PyInstaller is a “native compiler” for Python – much like py2exe, which I prefer for windows. For Ubuntu, I like install scripts, and source distributions: .tar.gz or .zip.

    Other files, to be installed at own risk, are distributed as tarballs where you have to perform more work in order to install them like sometimes compiling, and other times just configuring and running a simple install script (not by double-clicking it though).

    So, what is the practical security difference with a one-click installer (that installs malicious aswell as well-behaved code) and running a Python-interpreted script?

    In both cases, it is a matter of judgement stemming from the end-user.

  17. What one-click installer?

    OK, so the problem seems to be twofold:

    1. Should python files be run when double-clicking them (perhaps using the shebang)?

    Here, it might be a problem to know if a terminal needs to be opened first (for user interaction via CLI) or even if arguments to the script is needed. Therefore, I argue that this behavior could be problematic:

    a. Because if the program wants CLI interaction and you run it without a terminal, communication is lost. And if the script wants arguments, none are given.
    b. If you try to solve a. by always opening a terminal, it might seem confusing when running a stand alone GUI like a game or so.

    Therefore I think that python scripts (.py) should not be considered “runnable” without a shell/terminal. Maybe some other type of file, that could be a wrapper, that tells you “can be run without a terminal” like .pyw .

    2. Installation smoothness. When running Ubuntu, you normally install programs using the application manager or apt system. If you are to install third party apps from other sources, and they are not packaged as a .deb file, you need to take some actions in order to make the programs executable. This is more of a general “problem” and not so much a problem with Python on Ubuntu.

    I read your post as a critique of how Ubuntu handles double-clicking on python scripts.

    I think that the idea that double-clicking scripts that need an interpreter stems from windows users experience with windows (where it’s mostly double-clicking to get things done). But even windows has dealt with the problem of “opening a terminal (cmd.exe) or not?” by introducing two kinds of interpreters (for python and perl for instance – perlw and pythonw) one that runs scripts by interpreting them in a shell/terminal and one that silently executes them without a terminal (pythonw). This has to do with problem #1.

    But even if we try to solve #1 by introducing some kind of pythonw for Ubuntu that runs scripts without opening a terminal and in the background, should they be run when double-clicking them without setting the executable flag first? I think not. And this is #2. If you distribute scripts/applications as textfiles, you need to tell the user to set the executable flag first if she wants to start them by double-clicking.

    The distributor could go through the process of creating a deb file like so: http://ghantoos.org/2008/10/19/creating-a-deb-package-from-a-python-setuppy/

    A .deb file, you actually can double-click in order to install. You are then presented with the standard package manager GUI front-end (if you don’t run it in the command line like dpkg -i file.deb).

    Why is this safer then allowing files to be executable by default? I think it is because you are told that you are about to install stuff and given information on what and how. This is different than running code just when clicked actually.

    So, #2 seems to be related to the application distribution model of Ubuntu (and not so much to python) and that in turn has to do with the security model.

    So, does this mean that running downloaded python games a bad user experience for the Ubuntu user?

    That, of course, depends on what experience and expectations the Ubuntu user has. If the Ubuntu user comes from an environment, say Windows, where clicking the file in order to start it works fine, then of course the Ubuntu user will be disappointed as you were. But that is not to say that Ubuntu should work like Windows imho.

    And as I said earlier, no it has nothing to do with platform independence actually, since that is only the requirement of being able to run at all on more than one platform, and not a promise that you can use any window manager to start the application using the mouse 😉

    Phew, long reply this time 😀

  18. Long answer, yes. 🙂

    Creating .deb files is like learning a new programming language. I tried once, but gave up when I saw that the procedure was so technical I’d spend *a lot* of time learning something that is unnecessarily complicated (IMHO). Before there is an easy-to-use GUI frontend for .deb creation, possibly specifically geared towards Python projects, that is a dead end to me.

    You have a more narrow view of platform independence than me, and yes I can see the technical view you have. I use both Windows and Ubuntu on a day-to-day basis, and I don’t think running .py/.pyw files on double-click is such a security risc (it’s still run with user-privileges , not root) that it warrants the significantly lessened user experience.

    That said, maybe a compromise would be for .py files to be run *with* a terminal, and .pyw files to be run *without* one? It is very clear at least.

  19. “Creating .deb files is like learning a new programming language.” Yes, that’s why I’ve kept saying that this is a problem for the ones distributing the python games. It’s their problem 😉

    “I don’t think running .py/.pyw files on double-click is such a security risc”

    No, but making downloaded files executable per default is in my opinion. The debate in #1 above is different from #2.

    #1 Was: Is it good to make them runnable via doubleclick? And should there be a terminal in which it runs?

    #2 Was: Should files be executable just after downloading them? Or should one have to set the exec flag manually or running some kind of installation?

    I think, in terms of #1, that your suggestion “maybe a compromise would be for .py files to be run *with* a terminal, and .pyw files to be run *without* one” solves the problem of not knowing if a terminal is needed.

    I don’t think that should be determined on basis of file extension, though. So perhaps some other model.

  20. .. the problem of not knowing if a terminal is needed.

    I don’t know if it is possible, but my initial approach to this, in the original blog post seven miles up (heh) is that the “runner” of the .py script should check whether the python process outputs anything to stdout/err. In case it does, fire up a terminal/viewer window/something else that shows that output. If the python process does not output anything to stdout/err, don’t create any terminal. A well-behaved command-line python script should output usage-instructions if no command line arguments are sent in to the script. So no worries for the command-line-utility-folks! 😉

    I don’t think that should be determined on basis of file extension, though. So perhaps some other model.

    Could you elaborate on this? After all, .py and .pyw are quite popular extensions. Just like .deb (which I presume is not, in general, executable..?)

  21. Bra idé. Du borde alltså skapa ett sånt verktyg (kanske i python? haha) som kollar det (det kan ju finnas fall där det är omöjligt men ändå – jag tänker på att det kan finnas kvar debugutskrifter till stdout som inte används) och låta det köra python-filer!

    Just det verktyget måste ju vara executable men det är ju inget konstigt.

    Me:I don’t think that should be determined on basis of file extension, though. So perhaps some other model.

    You:Could you elaborate on this? After all, .py and .pyw are quite popular extensions. Just like .deb (which I presume is not, in general, executable..?)

    In GNU/Linux in general, things are not decided on basis of file suffix but on a list of magic significants of files, usually found here:

    /usr/share/file/magic

    You can convince yourself using the command file and investigate the file above:

    $>file /usr/share/file/magic
    /usr/share/file/magic: magic text file for file(1) cmd

    So it’s a text file! Now open it an look for python in it! That’s the place for a new defenition “python file to be run without terminal” 😉

    I don’t think it’s a good idea to invent a new way of doing things, like looking at the file suffix!

    Basing actions on file suffices is also kind of problematic because it allows for people to trick users that a file isn’t what it is. Etc.

  22. file about file! Fascinating, as always when you dig deeper in the *nix ecosystem haha!

    So if a .py file is somehow magically categorized according to the file/magic convention, I could write a script (in Python, why not bite the tail when it’s possible?) to solve my little nagging problem, you say?

    Thanks for the discussion today mjp!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s