[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

The State of Web Browsing in Emacspeak [very long]



If you are reading this in the archives, please follow the thread to the
end.  

I have written this article for the archives.  Hopefully someone will
find it informative and get enough out of it that it makes it worth
writing.

Please send me comments.  If you feel that anything is missing or wrong,
if you think anything is not exactly clear, if you think you have a
better example or anything else feel free to let me know via the mailing
list.  

I do have a couple of questions that hopefully someone will help me out
with.  The use of these commands is not exactly clear to me:

emacspeak-w3-curl-url-under-point
emacspeak-w3-url-rewrite-and-follow
emacspeak-w3-url-expand-and-execute
emacspeak-w3-xslt-filter

There are actually more, but these seem that they might be useful to
end-users.  

Article follows.

The State of Web Browsing in Emacspeak

* Introduction

Web browsing in emacspeak can be made as simple or as complicated as
you wish to make it.  On the one hand, all that need be done is to
open a web browser and fetch a particular url.  This works fine in
many cases with exceptions being things like html tables.  On the
other hand, if you spend a little time learning a particular browser
and its functionality, web browsing can be made more efficient and the
web more navigable.

This document is intended to be an introduction to the 2 primary
browsers that run under emacs: emacs/w3 and emacs-w3m.  I also intend
to introduce a couple of add-on packages intended to make life easier
and to address specific shortcomings.

The primary reason for this documentation is the fact that a lot of
the documentation of the various functions tells exactly what it does,
but not why or in what circumstance one might use it.  Hopefully I can
remedy that here.

** Who this document is intended for

While this document is geared toward emacspeak users, it might be
helpful for other users as well.  I have tried to note where functions
are specific to emacspeak.

** Assumptions

This document assumes emacs is installed.  If reading the section on a
particular browser and trying out the associated functions, it, of
course, assumes that the browser is installed and working. For
features specific to emacspeak, a working install of emacspeak is
necessary.  It is far beyond the scope of this document to help with
the installation of these programs.

For help with installing any of the above bits of software, the
following mailing lists and / or newsgroups are available:

emacs - gnu.emacs.help
emacs-w3m - see http://emacs-w3m.namazu.org
emacs/w3 - the mailing list for w3 is pretty dead.  Sorry. 
The documentation is, however, not terrible.
emacspeak - see http://emacspeak.sf.net

It is highly suggested that the mailing list archives, google, and the
relevant documentation be consulted before posting messages to any
mailing list.  Nothing is more irritating than answering the same
questions over and over.  Those of you with kids know what I am
talking about.

It is also assumed that the reader is comfortable with using emacs
itself.  Understanding the convention for communicating keystrokes to
run commands, navigating documents, and the like are not covered.

* Emacs-w3m

emacs-w3m is a front-end, written in emacs lisp, to the text-based web
browser w3m.  It supports tables, images, and all major web protocols.
It is also fully supported in emacspeak.

** advantages

The main advantage of emacs-w3m is speed.  The rendering of web pages
is done with the external program w3m.  which is written in c, so it
is fast.  It also seems not to choke on some web pages like the
alternative, emacs/w3.

Another big advantage to emacs-w3m is that it is under active
development.  Problems seem to be fixed soon after they come to the
attention of the developers and are available very quickly as long as
you are willing to use the cvs version.

For those who have limited sight as opposed to no sight at all,
emacs-w3m displays all graphics within its own buffer.  This might be
a misconfiguration on my part, but there are some images that open in
an external window when opened from emacs/w3.  

I can also say that I cannot remember a page that wouldn't render in
emacs-w3m.  This is not something I can say about emacs/w3.

** disadvantages

Taken by itself, the disadvantages of using w3m are few.  Long periods
between releases can be seen by some to be a disadvantage although the
alternative browser's time between releases is now several years, I
believe.  

With hardware or software that support multiple voices, such as IBM's
Via Voice, one can allow differences in speech to indicate differences
in document text.  For example, headings can be spoken in a voice with
a lower pitch, links in a higher voice.  The voices in emacs-w3m are
much more limited than those in emacs/w3 and not all textual
attributes can be indicated.  As of the writing of this document, pre
tags and the like cannot be indicated.  It has been within the last
few months that italic text support has been added.

Another disadvantage is the fact that the rendering in emacs-w3m is
done by the external process, w3m.  Since w3m is written in C, it is
not as easily customizable from within the community of emacs users.
Code written in emacs lisp can be substituted for code in a running
emacs instance, something that is not possible in C.

A further disadvantage is the lack of table navigation, i.e. the
ability to move from cell to cell in a table to more easily understand
the information presented.  Linearizing the table is available but
sometimes it is difficult to keep the meanings of the columns in your
head to discern the meaning.  Maybe it is just me.

A disadvantage shared between the browsers is the lack of support for
javascript.  Many sites use javascript for links to the next page,
such as Reuters web site.  This can sometimes be worked around using
the google transcoder.  See below for more information.

** emacs-w3m native functions

This part of the document is not meant to replace the describe-mode
information available via the "C-h m" keystroke or the info manual.
This is by no means an exhaustive list of available commands.

emacs-w3m has considerable functionality and many features that
make browsing more efficient and convenient.  Below are many of the
functions that are native to emacs-w3m.

Cap "S" will query for a search term.  By default, the search engine used
is google.  This can be changed with a prefix command.  To see the
available engines, after hitting C-u S, when it asks for the engine
hit tab to get a list of available completions.  Tab completion of
partial words works here as well.  For example, hitting "y" and then
tab will complete to the word "yahoo".

Form navigation can be achieved with the right and left brackets.
This is useful for times where you want to go to the input field on a
web page without having to tab through the links.  Google is a good
example of where this would be useful.  The right bracket moves point
to the next form, and the left bracket moves to the previous, wrapping
around at the ends of the buffer.

While in a form, using C-c C-c will submit the form with no need to
tab around looking for the "Submit" button.

** emacspeak specific functions

Many extensions have been written by the emacspeak community to expand
emacs-w3m and make it more accessible.  Below are those extensions.

While being a variable and not a function, the variable
emacspeak-w3m-speak-titles-on-switch makes emacs-w3m more friendly.
It tells emacspeak to speak the document title when switching between
emacs-w3m buffers.  The default, nil, will speak the mode line.  This
is the same behavior seen in all other buffers.  This speaking of the
document title is more descriptive than just hearing *w3m*<4>.  This
variable can be customized in the usual ways, either in your .emacs
file or via the Customize interface.  If you do set this variable,
reading the mode line can still be achieved with C-e m.

The "j" command will jump to the first occurance of the title in the
document.  Each time you use this command in succession, it will jump
to the next occurance of the title.  This is very convenient for web
pages that have the title show up in multiple places in the document.
The web site for The Register is a good example of this.

The "l" command uses the default media player to play the media stream
at point.  I believe this is set to emacspeak-m-player by default.
While beyond the scope of this document to describe the installation
of mplayer, I would highly advise using this media player.  With the
correct codecs installed, mplayer will play everything you will find,
I believe, with one convenient player that does not require a GUI.

"C-t" will run the current page through the google transcoder.  The
transcoder is very useful to the users of screen readers as it takes
care of formatting issues like tables.  While not usually a problem,
There is a function to linearize tables in emacs-w3m, but it is not
always successful.  You can reverse this command by giving a prefix
argument.

"M-t" runs the url under point through the transcoder.  This command can
be reversed by giving a prefix argument.

Extensions have been written to take advantage of much of the google
functionality.  The following describes how to access many of these
extensions.  

"C-c C-g" will search the current web site with google.

"C-c C-x" will extract the current page from the google cache.  A prefix
command will extract the url under point from the cache.  This is
useful if, for instance, a site is down for one reason or another and
you absolutely must have the information.

"C-c C-l" will find pages similar to the current page.  This is just
like the "similar" link at the end of a google search result.

"C-c C-r" will browse the rss link under point.  One reason this might
be useful is to first try out the rss feed before you configure your
rss reader to subscribe to the feed.  For a easy to use rss reader,
see the section below on emacspeak-rss.

The letter "x" is the prefix key for the xslt transforms.  xslt is
some powerful ju-ju.  It takes a web page and does all kinds of cool
things with it.  These things are listed below.

The command "x a" will apply an xslt transformation to the current
page.  one particularly useful transform is to linearize tables.  This
takes the tables and formats them linearly down the page as if they
were paragraphs instead of table cells.  This command can be more
easily applied with the "x l" command.

If you want to always have tables linearized, use the "x a" command to
select linearize and then the "x o" to toggle xsl on.  This will
linearize the tables in every subsequently visited page for this
session.  While it seems possible to make this happen automatically in
emacs/w3, I do not believe this is currently possible in emacs-w3m.
I, personally, don't see this as a problem as there are many pages
which don't need to be linearized and this just creates unnecessary
overhead.  To each his own, I guess.

The "x b" command will add a submit button to forms that do not have
one such as the one on the emacswiki.  Submitting forms can be more
easily achieved with "C-c C-c".

If on a google search page, the "x h" command will give you only the
search hits.  An easier way of getting the same results is to use the
url-template "Google Hits" which returns the same results.  Another
advantage of using the url-template is that it can be done from
anywhere, not just in an emacs-w3m buffer.

The "x t" command sorts the tables.  It takes the nested tables and
unnests them.  While sometimes useful, this is not nearly as useful as
the same function in emacs/w3.  While doing the same thing, emacs/w3
has the capability of allowing the user to navigate the tables cell by
cell.  

** various tips and tricks

I have nothing here.  If anyone can think of tips and tricks for
getting around annoyances and the like in w3m, let me know.

*Emacs/w3

emacs/w3 is a web browser written completely in emacs lisp.  It has
some really nice features applicable to the emacspeak community such
as the ability to navigate tables and support for the w3c's aural
cascading stylesheets.  

** advantages

As mentioned above, the ability to navigate tables is a super help.
emacs/w3 also has support for cascading stylesheets.  This allows
incredible control of voices used for what would normally be visual
attributes of the text such as bold, italics, preformatted text and
the like.

Another advantage of emacs/w3 is that it is written completely in
emacs lisp.  With some effort, emacs/w3 is very customizable and quite
extendable.  

** disadvantages

Rendering can be slow.  Sometimes it can be painfully, excruciatingly
slow.  That might be a slight exaggeration, but slow it is.  This is
because it is written in lisp... something I mentioned just above as a
strength.  It is a trade-off, but one that some see as worth it.

Another disadvantage is that emacs/w3 chokes on some pages.  Sometimes
it gives error messages and doesn't display anything.  Sometimes it
does this to some people and doesn't do it to others as we saw on the
emacspeak mailing list a short time ago.  Sometimes it gives error
messages and renders the page anyway.

As mentioned in the emacs-w3m section, this browser does not support
javascript, which is a problem usually for links on some sites.  One
way this problem can be sometimes worked around is to transcode the
link under point via google.

One other major annoyance of emacs/w3 is that sometimes it simply
stops doing anything while rendering a page.  There is a way I have
found to get around this.  I hit "C-g".  I usually wait five to ten
seconds and then simply quit, using "C-g".  Not always, but usually,
the page has already completed downloading and is being rendered and
it therefore is not a problem.  

emacs/w3 also has no bookmark functionality.  This can be remedied in
several ways.  One simple way, mentioned below, is to use the emacs
package bmk-mgr.  Another way is to use org mode with remember which
is the method used by Dr. Ramen, the author of emacspeak.  

History back and next in the browser also seem to be broken but this
is not generally a problem for me as I never look back.

** emacs/w3 native functions

Many functions for efficient navigation of the internet are native to
emacs/w3.  Some of the most useful functions are listed below

Unlike emacs-w3m, the information presented by describe-mode is very
complete, but a little terse.  All the function names are listed and
asking for help on particular functions works well.  That being said,
use this list to augment, not replace, the built-in help that is
available in emacs.

"C-f" will open a new buffer containing the cell point is in.  In most
instances, this works very well.  Imagine you are looking at a page
that is divided into 4 distinct areas: a cell at the top of the page
that contains a banner and some navigation, and a "body area that is
divided into three sections consisting of more navigation, an article,
and advertisement.  If point is in the main article cell, using C-f
will open another buffer that contains only the text of that cell, the
article you are interested in.

One caveat is that this does not always work as advertised.  Sometimes
the leftmost character of each line is missing.  At least it makes for
interesting reading.  Usually when I have this problem I simply exit
that buffer and linearize the tables in the original page.

The "m" key executes a very useful command.  It will complete a link
on the page.  Imagine that you are reading through a document and you
hear a link that you need to visit.  You could tab through all the
links until you hear the one you want or you could hit the "m" key and
enter the link text at the prompt.  Completion is available and it is
not case-sensitive.  Efficient, no?

The period in a cell will speak the contents of that particular cell.
This command is, in my opinion, most useful when navigating tables
with cells that have only one paragraph or less.  I tend to not read
whole articles in this manner because, inevitably, someone will
interrupt me and I will lose my place.

The equals key, while in a table cell will give you the cell
information.  It tells you the row and column position, the size of
the table, and at what nesting level the table is.

The pipe key, is used to read the table column.  As this command seems
to read the rectangle the column is in, this command is most useful
when used in a table where there is no column spanning, i.e. all rows
and columns are uniform.  

Here is a list of table navigation commands:

C-e + moves to the beginning of the next table row.
C-e - moves to the beginning of the previous table row. 
C-e < moves to the beginning of the table.
C-e > moves to the end of the table.
C-e = moves to the top of the table column.
C-e <down> moves to the next cell down in the same column.
C-e <up> moves to the previous cell in the same column.
C-e <left> moves to the previous cell in the same row.
C-e <right> moves to the next cell in the same row.

As you can see, table navigation in w3 can be easy and fun.

** emacspeak specific functions

An incredible amount of work has been done by the emacspeak community
to make emacs/w3 accessible to those with visual impairments.  Here is
an explanation of some of those functions.

The command C-t will toggle the visibility of table borders.  This
command might be useful where you want to hear all punctuation symbols
on a page but the table characters get in the way.

The quote key will execute a command that allows you to skim the
contents of the buffer.  it will read the page,
paragraph-by-paragraph, pausing between paragraphs to prompt you to
move on by pressing the spacebar.  If you hit the spacebar in the
middle of a paragraph, it skips to the next paragraph.

Another skimming command is bound to the "z" key.  This will allow you
to zip through web pages by logical blocks such as div, paragraph, and
table tags.

Using the imenu facilities is another way of skimming the document and
getting to the information you desire.  imenu works especially well
for well-structured documents.  The first thing that need be done is
to copy the w3-imenu.el file from the contrib directory of the w3
directory to somewhere in your load path.  I am using the cvs version
of w3 and my w3 directory is under /home/rdc/sourceforge.  The easiest
thing to do is probably to do an M-x locate and search for w3-imenu.el
to see where it is.  After locating the file, move it into your load
path.  In my case I have it under /home/rdc/share/emacs/site-lisp/.  

There are two ways to use the imenu facilities: automatically and
manually.  Since I do not use imenu on every site, I prefer to invoke
it manually to save the time required to build the index.  

Once things are in place, invoke imenu with the "j" key.  This will ask
you for an index position.  Hitting the tab key will give you a list
of the possible index positions.  Another way of navigating the
document would now be to use the keys M-n and M-p to go to the next
and previous index positions.

Cap "A" and cap "R" serve the same function.  Cap "A" browses the Atom
feed at point and cap "R" browses the rss feed at point.  This is
useful to sample the feed, so to speak, before going through the
trouble of configuring your feed reader to fetch the feed.  It might
also be useful to grab the headlines from a page and present them in a
more concise, readable format.

Google provides many useful tools for web surfers.  The following
commands are useful to access much google goodness.

Cap "C" extracts the current page from the google cache.  With a
prefix argument it will extract the link under point.  This is useful
for those times when a particular site is down... maybe it is in the
cache... maybe it is not.  It can also be used for when particular
pages are removed from a site like in the case of a government
conspiracy.  Are we at war with Eurasia or East Asia?

The slash key will search google for pages similar to the current
page.  

The command "g" will do a google search restricted to the site of the
document.

The "l" command googles for who links to this page.

The command "t" runs the url under point through the google
transcoder.  This is useful for sites that are heavy on the use of
tables and the xsl transforms are not helpful.  It also works on some
sites that use javascript to go to the next page in the story, such as
Reuters.  Using a prefix argument with this command will untranscode
the url under point for pages that are currently transcoded.  

Cap "T" will jump to the first occurance of the title in the
document.  Multiple consecutive executions of this command will jump
to further occurances.  This command is probably one of the most
useful timesavers while web browsing.   

M-s jumps to the submit button for the form you are editing.

M-r plays the media stream at point with the default media player. 

Saving the best for last, "e" is the xsl map prefix.  As I mentioned
in the section on emacs-w3m, xsl transforms are some powerful magic
that takes a web page and transforms it in some way.  Linearizing
tables is a good example, and the one I use most often.  

The keystroke "e a" prompts for an xsl transform to apply to the
current page.  If you know the name of the particular transform you
want you can use tab completion to select it.  Otherwise, you can hit
the tab key to get a buffer that contains the list of choices.

If you know that you want a particular transform done automatically
you can use the command "e s" to select a transform and then "e o" to
turn xslt on (the same command will turn xslt off).  Then, every page
opened from that point on will have the transform applied.

There is the variable emacspeak-w3-xsl-transform that can be set via
the usual methods.  This variable specifies a transform to use before
displaying a web page.  As I mentioned above with emacs-w3m, I do not
have transforms applied all the time as they are not always
necessary and create overhead and a delay in display of pages.  This
is especially important with emacs/w3.  Maybe I am just too
impatient. 

Sometimes it is just easier to read the printer friendly version of a
story instead of having to linearize the tables and search for the
content.  Also, some sites, like the New York Times, I believe, make
you navigate several pages to read the whole story, but if you select
a "Print this story" link you can read the entire story on one page
formatted without a lot of the cruft on the normal page.  This is
where the "e P" command comes in.  It will extract all the print
streams from the current document.

Closely associated commands are "e r" and "e Cap R" which extract the
media streams from the current page and from the link under point,
respectively.  

** various tips and tricks

As I mentioned above, using "C-g" when it seems the browser is not
responding will often display the page with no ill effects.  Your
mileage may vary.  Taxes, tags and title are extra.

Another useful tip is the use of the "k" key.  This key will place the
current url in the kill-ring for later yanking.  If a page will not
render correctly, using "k" will get the url and allow me to pass it
to emacs-w3m.  The counterpart to this command, Cap "K" puts the url
under point in the kill-ring.

* Add-ons

Some of these are emacspeak specific, some are not.  You can usually
tell by the name.

** emacspeak-url-template

I love this package.  Since changing my primary browser to emacs/w3 I
have really been giving the url-template package a workout.  The
url-template package contains templates that prompt you for
information to supply to various sites to retrieve information without
all the fuss of having to go to the site and navigate it.  One really
nice thing about url-templates is the fact that they need not be web
pages.  Media streams can also be made into url-templates.  

The way to get to the templates is with the command "C-e u".  A tab at
the prompt will give you a list of the available templates.  You
should go now and have a look at the info manual section on
url-templates and read it.  I'll wait here.

By the way, the ones I find most useful are the "Google Hits",
"emacswiki search" "NPR On Demand" and "Weather forecast from Weather
Underground".

** emacspeak-atom and emacspeak-rss

These are fairly simple rss and atom browsers for the emacspeak
desktop.  Using the Customize interface you add feeds in the form of
titles and urls.  Then you call the readers with "C-e C-u" for rss
feeds and use M-x emacspeak-atom-browse for atom feeds.  There is also
emacspeak support for newsticker, an rss / atom reader that is a part
of emacs 22, but I have never used it.

I personally use gnus for rss feeds but setting gnus up for only that
purpose is like hunting rabbits with a bazooka. 

** emacspeak-websearch

emacspeak-websearch provides more search options than you can shake a
stick at.  A lot of this functionality is duplicated in url-template
but some is different.  emacspeak-websearch provides search for
dictionaries, news sites, software sites, google tools, weather,
currency converter and much more.  It can be accessed with the
keystroke "C-e ?".  At the prompt, you can enter another question mark
to get a list of the available search options.  You will then be
prompted for the necessary information.  One of the nice things about
this package is that, when using w3 it attempts to jump to and read
the most relevant information on the result page.  

See the info manual for more information.

** bmk-mgr

This is a newcomer to the emacspeak world.  In the interest of full
disclosure, I am the one that wrote the emacspeak module that makes
this package accessible.  It is a bookmarks manager that works with
both emacs/w3 and emacs-w3m.  As of the writing of this document there
are still some issues, especially when using it on emacs version 22,
but those are being worked on.  I think this is a good solution to the
problem of emacs/w3 not having bookmarks functionality.  See
http://www.emacswiki.org/cgi-bin/wiki/EmacsBmkMgr for more
information. 

* Conclusion

emacspeak makes the internet not only accessible to those with visual
impairments, but it makes browsing and information retrieval quick and
efficient.  If a user will spend a little time up front to learn the
tools available to access the web, the increase in efficiency and
ability will more than make up for the time spent.  The nice thing
about these tools is that you can integrate them in your day-to-day as
you have the time.  While it is not necessary to use everything mentioned in
the above document, if you add some of these tools to your repertoire
you will not be sorry.  

In the end, no one makes you use a hammer to drive nails but it sure
beats using a banana.


-- 
Robert D. Crawford                                      rdc1x@comcast.net

Place stamp here.

-----------------------------------------------------------------------------
To unsubscribe from the emacspeak list or change your address on the
emacspeak list send mail to "emacspeak-request@cs.vassar.edu" with a
subject of "unsubscribe" or "help"



If you have questions about this archive or had problems using it, please send mail to:

priestdo@cs.vassar.edu No Soliciting!

Emacspeak List Archive | 2007 | 2006 | 2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 | 1998 | Pre 1998

Emacspeak Files | Emacspeak Blog | Search the archive