Structure editors

[AndrewAPrice]

Why would you have pictures embedded into source code? I was talking about using colours and graphics to represent the source code, but there's no reason you couldn't use sound instead.

[iansjack]

You can really apply structured editing to any form of text. I must admit that I was intrigued to gain some insight into how blind people use computers in general and particularly where pictorial information is involved. But if that's going off topic, let it be.

[onlyonemac]

@onlyonemac - What sort of feedback would your ideal structure editor give you? Could you write out an example of what you'd like to hear as you're editing and navigating your code?

Well say I have some C code like this:

Code: Select all

int main(int argc, char *argv[])
{
if (argc == 1 && strcmp("parameter", argv[1])
{
printf("Parameter was given, name is %s.\n", argv[0]);
exit(1);
}
else
{
printf("No parameter, or different parameter, given.\n");
}
exit(0);
}

I would want to hear something like this, assuming that I have just opened the file and am currently positioned on the first line:

Code: Select all

<press key to go to next line>
"collapsed block, if"
<press key to enter block>
"function printf with 2 parameters"
<press key to enter parameters>
"parameter 1 of 2, string"
<press key to read parameter value>
"Parameter was given comma name is percent s dot backslash n"
(at this point one may press a key to edit the parameter, which would behave like a single-line text editor)
<press key to exit block>
"collapsed block, if"
<press key to enter parameters>
"parameter 1, conditional"
(at this point one may press a key to edit the condition of the if statement, which would behave like a single-line text editor)
...and so it goes on...

Obviously I would also have room for changing the settings, e.g. I might want to not split the parameters and the function names depending on what I am doing, so like if I am trying to glance over some code I will want the parameters read out to me as I go. But it would still be "intellegent", by interpreting the parameters and stating their type and splitting them into a neat list rather than reading them out with the separating quote marks, commas, etc.. Another thing would be to replace common statements with meaningful sounds ("audio icons", a concept of which I am a great fan despite it being under-used). Overal it's an interesting project, but not something that is high on my priority list despite the fact that it would be useful to me even now.

What about pictures. How do you convert them to audio so that a blind person can recognize them?

In short, you don't. I've been thinking of a few ideas based around the idea of SSTV, but it would take a lot of training to use and not be very effective. Pictures are not usable to say the least, and a well-made webpage should include alternative text to describe the picture to screenreader users but most of the time they don't.

[iansjack]

Pictures are not usable to say the least

That's a bit puzzling.

I must have misunderstood you as I know that you are able to interpret pictures on this website.

Anyway, it's getting too far off the main topic.

[AndrewAPrice]

Thank you, onlyonemac. It's incredibly insightful to hear how your ideal editing environment would work. I feel that a structured editor would work very well because it would be more tree driven and easily navgatable via keyboard than free form text.

Just out of curiousity? Have you ever used a refreshable braille display? They are very expensive, but I wonder if blind programmers would be more productive if they were able to read a line of code in braille than it spoken to them. Is reading braille relatively fast or slow?

I think there would be a huge market if a company was able to develop an affordable braille display. I also think there would be a huge market for video games for the blind.

[embryo2]

onlyonemac

Really interesting discussion shift!

I also want to exercise my mind in the field of alternative perception

I just don't know enough about the devices that help blind people, but anyway it would be good to discuss such user interface. Are there some kinds of a "blind screen" with rods pushed and pulled at a grid crossings? If yes, then what do you think about it's ability to help a blind person? And not only with the braille encoded signs, but with a full spectrum of tactile image representation. For the programming language editor it can "display" something like text line length, represented by a line of pushed rods with the length of the line proportionate to the length of the text line. It can help to distinguish short lines (like block start and end which is represented with curly braces) from long lines (the actual program code). Also the line length difference can give a tactile picture of the actual text and I think it is possible to use such representation as a map where your hands can feel text area borders, can discern language blocks, can percept a pattern of text lines and help to understand the actual location in the text (like first goes short variable declaration, next goes another short line with formula and next goes a long line with function name and parameters, here all those lines together represent a pattern that may be can be recognized by hand(s) and as a result can help to "see" the current position in a program). Also it is possible to place a scroll wheel on a side of the "display" and when scrolled the rods are pulled and pushed just like pixels on a standard display are highlighted or become dark. And of course, another mode of the display can "show" the braille signs (like when you feel the required line under your hand you can push the rods and it will switch the display mode to the braille representation of the text line).

But may be it's already done and I just reinvent the wheel? It would be very interesting to see your answer!

[onlyonemac]

I must have misunderstood you as I know that you are able to interpret pictures on this website.

I still have a small amount of useable sight, but it is not going to last long.

Just out of curiousity? Have you ever used a refreshable braille display? They are very expensive, but I wonder if blind programmers would be more productive if they were able to read a line of code in braille than it spoken to them. Is reading braille relatively fast or slow?

I have not used a refreshable braille display but I am really keen on the idea. I'm not sure how good it would be for programming because punctuation characters are rather clumsy in braille (most of them need two or more braille symbols to represent the single punctuation character) but again the structured editor would help in this case as the punctuation characters (which do, to be honest, make the screenreader more difficult to understand as well) are made transparent to the user and would never need to be dealt with. However I do get tired of listening to long webpages, and I have read "printed" braille (braille embossed onto paper, that is) and I do prefer it to listening. (Unrelated, but I am also not a fast braille reader - I have not been reading for that long. Nevertheless it is possible for one to read braille at least as fast as print, and on average a "good" braille reader is faster than a "good" print reader.)

I also think there would be a huge market for video games for the blind.

Absolutely! I've had a few ideas for such games myself...

Are there some kinds of a "blind screen" with rods pushed and pulled at a grid crossings? [...]

That is a good idea that I have considered before as well; it has not been done. The main issue, I believe, is that the technology is not quite there yet. For example, a multi-lline braille display is already pushing the limits of the available "resolution", and such a device is a bulky, heavy and expensive piece of equipment due to the number of moving parts. So while the idea is very good, practically we don't quite have the technology yet for a nifty little tablet-like device with a tactile screen.

Nevertheless I have thought about such a device before and I like the idea and can imagine it being very useful and user-friendly. One interesting that you have pointed out though which I had not considered before is the idea of a "zoomable interface" whereby one can switch from skimming over the lines of code to actually reading the code in braille. Of course this improves useability greatly not only for code but also for other tasks involving text, even reading a college textbook. Print readers have always been able to glance down a page to find their place by looking at the "shape" of the text, and while an experienced braille reader is sometimes able to do so with "printed" braille by e.g. brushing their fingers down one margin or the other, that isn't really feasible when working at a computer and one of the greatest limitations of a computer interface for the blind is that only one line of text can ever be read/listened to at a time. I had thought previously of using such a tactile screen to represent a kind of GUI, but the zoomable interface is, I think, in fact a much more effective use of the technology, and would also work more effectively than a tactile GUI on a lower-resolution display.

However the device would also have important uses beyond just reading, as for example tactile diagrams, while they do exist and are an effective system of representing diagrams such as maps (very useful!) for blind people, are not widely available and are difficult to produce, but could be represented quite easily on such a device. And, of course, the mention of video games for the blind ties in nicely as well, as many games benefit from some sort of spacial layout and while 3D sound is one way of representing a space for a blind person, a tactile display could be useful in other games (sighted people don't always play first-person 3D games, so why should blind people be restricted in such a choice?).

Needless to say, the device could be complemented with audible output, and with a user-switchable option one could even have a system whereby selecting a line of code (by double-tapping it or something, I guess, as is the standard gesture for selecting something on a normal touchscreen device using a screenreader) would not switch to reading or editing it in braille but in fact reading it out audibly and allowing the person to edit it audibly. That would provide a lot of flexibility in how the user interacts with the device and the content, as some blind people prefer audible information and some (myself included) prefer tactile information.

[embryo2]

Are there some kinds of a "blind screen" with rods pushed and pulled at a grid crossings? [...]

That is a good idea that I have considered before as well; it has not been done. The main issue, I believe, is that the technology is not quite there yet. For example, a multi-lline braille display is already pushing the limits of the available "resolution", and such a device is a bulky, heavy and expensive piece of equipment due to the number of moving parts. So while the idea is very good, practically we don't quite have the technology yet for a nifty little tablet-like device with a tactile screen.

I have googled a bit and found "affordable" 40 cell display for 2700$. Roughly 70$ per cell. But the traditional keyboard also includes a lot of moving parts and it's price is about 10$ (9c per key). Something is wrong with this market. May be there actually is no viable demand for such devices? Because audible output is cheap. Else the price just should go down and mechanical complexity here is not a limiting factor.

One interesting that you have pointed out though which I had not considered before is the idea of a "zoomable interface" whereby one can switch from skimming over the lines of code to actually reading the code in braille. Of course this improves useability greatly not only for code but also for other tasks involving text, even reading a college textbook.

Then we should create it. At least it is possible for us to write a driver for it

In fact there are many solutions beside of the piezo crystals that can move rods. But is there a viable market for such device?

And, of course, the mention of video games for the blind ties in nicely as well, as many games benefit from some sort of spacial layout

Yes, "blind games" with moving waves under your fingers can attract even non-blind people. All smartphones now have haptic motor, so why they can't have a small blind screen? The smartphone's big market can decrease the price greatly.

Needless to say, the device could be complemented with audible output, and with a user-switchable option one could even have a system whereby selecting a line of code (by double-tapping it or something, I guess, as is the standard gesture for selecting something on a normal touchscreen device using a screenreader) would not switch to reading or editing it in braille but in fact reading it out audibly and allowing the person to edit it audibly. That would provide a lot of flexibility in how the user interacts with the device and the content, as some blind people prefer audible information and some (myself included) prefer tactile information.

And it is really the great target for the structured editors. There is no limitation of the myriads of existing text oriented editors/viewers. The space is empty and open for creativity.

[onlyonemac]

I have googled a bit and found "affordable" 40 cell display for 2700$. Roughly 70$ per cell. But the traditional keyboard also includes a lot of moving parts and it's price is about 10$ (9c per key). Something is wrong with this market. May be there actually is no viable demand for such devices? Because audible output is cheap. Else the price just should go down and mechanical complexity here is not a limiting factor.

Quite simply yes, there is a problem with the market. In short, the problem is that the manufacturers exploit the fact that there are not a lot of people who need braille displays, but those who really do need them are forced to pay the price as they cannot do without. It was the same with screenreader software - the JAWS screenreader costs about $1000 - until the free NVDA screenreader came along...

Yes, "blind games" with moving waves under your fingers can attract even non-blind people. All smartphones now have haptic motor, so why they can't have a small blind screen? The smartphone's big market can decrease the price greatly.

I - and, I believe, other people - have considered the use of a smartphone's vibration functionality. While such functionality is useful for providing feedback of various kinds, it is not useful in the sense of tactile displays because it is either "on" or "off"; you cannot represent texture with it, and one cannot "feel" it, it is more like having the device produce a sound whenever the finger is over an "active" part of the display but remain silent when over an "inactive" part. Using the vibration function in such manner doesn't really feel like a tactile display.

[AndrewAPrice]

I think braille displays are very expensive because of what onlyonemac said. They are only feasible as something a company with plenty of money would purchase to increase their employee's productivity. Or if you loose your sight in an accident and somehow sue your medical insurer that it's necessary for you to continue your work.

I'm thinking of the feasibility of a low cost braille display. At 6 dots per character, with 80 characters that's 480 dots. Each dot could be powered by an electromagnet that pushes it up, and when unpowered, gravity causes it to drop. Maybe one day we will see it cheaper than $100.

[onlyonemac]

I'm thinking of the feasibility of a low cost braille display. At 6 dots per character, with 80 characters that's 480 dots. Each dot could be powered by an electromagnet that pushes it up, and when unpowered, gravity causes it to drop. Maybe one day we will see it cheaper than $100.

I don't want to try to state a "reasonable" price for a braille display, but I definitely think that at least a part of the high price is due to exploitation of the target market. Also there are a few issues with your statement: 1) it is necessary to piezoelectric crystals as per the current designs as a large number of electromagnets would consume a ridiculous amount of power 2) a braille display is normally around 40 characters long; an 80-character display is considered "large" and 3) most braille displays have 8 dots per character, with the extra dots optionally used to signal various formatting attributes of the text rather than using separate characters up for the not-so-important formatting as is the case with "printed" braille.

EDIT: It seems that there are technologies that are cheaper than piezoelectric crystals: https://en.wikipedia.org/wiki/Braille_e-book.

[embryo2]

In short, the problem is that the manufacturers exploit the fact that there are not a lot of people who need braille displays

I hope that with the lower price much more people will buy such device and increased market will make the price even less.

I - and, I believe, other people - have considered the use of a smartphone's vibration functionality

Actually I was talking about tactile screen, when the surface can be changed. It can add more entertainment capability to smartphones for non-blind people and the ability to read in a much more convenient manner for blind people.

[onlyonemac]

I - and, I believe, other people - have considered the use of a smartphone's vibration functionality

Actually I was talking about tactile screen, when the surface can be changed. It can add more entertainment capability to smartphones for non-blind people and the ability to read in a much more convenient manner for blind people.

OK so what you're saying is that it's worthwhile adding a tactile screen to the average smartphone because sighted people can benefit from it as well? In that case, what does the mention of the vibration function have to do with anything?

EDIT: Never mind I get it. Well no, because I still don't see how your mention of the vibration feature relates to anything, but basically what you're TRYING to say (somehow) is that because phones have vibration functions then it is feasible for them to have a tactile screen. I don't get the logic in that, but your other argument that sighted people would benefit from a tactile screen thus making it worthwhile is much more convincing, despite being debatable.

[Rusky]

There is at least one existing product for pressable buttons on touch screen keyboard keys: http://tactustechnology.com/technology/, http://www.getphorm.com/. I don't know how well it would handle braille in its current state- it probably has "button" size and number limitations since it uses a fluid to raise parts of the screen.

[onlyonemac]

There is at least one existing product for pressable buttons on touch screen keyboard keys: http://tactustechnology.com/technology/, http://www.getphorm.com/. I don't know how well it would handle braille in its current state- it probably has "button" size and number limitations since it uses a fluid to raise parts of the screen.

That part of the technology is simple. LCD displays and touch screen surfaces are already flexible enough to have a limited amount of movement above their plane of rest. The difficult part is producing a tactile screen with a decent resolution, no matter whether or not it has an LCD and/or touchscreen over the top of it. As I have previously mentioned, there is a developing technology along these lines which could easily be integrated into a smartphone or tablet https://en.wikipedia.org/wiki/Braille_e-book.