COPTR - User contributions [en-gb]

GNU Wget

2014-10-01T03:54:09Z

Danielle plumer: Added information about features, license, platforms & installation, documentation, and user experiences. Fixed Ohloh ID.

{{Infobox_tool
|purpose= Non-interactive network downloader
|image=Gnu2.png
|homepage=http://www.gnu.org/software/wget/
|license=GNU General Public License
|platforms=Unix, Linux, Windows, Macintosh
}}


[[Category:Web Crawl]]

= Description =
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

== Features ==

From the Wget manual:

* Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.
* Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.” While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.
* File name wildcard matching and recursive mirroring of directories are available when retrieving via FTP. Wget can read the time-stamp information given by both HTTP and FTP servers, and store it locally. Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. This makes Wget suitable for mirroring of FTP sites, as well as home pages.
* Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.
* Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. Wget uses the passive FTP downloading by default, active FTP being an option.
* Wget supports IP version 6, the next generation of IP. IPv6 is autodetected at compile-time, and can be disabled at either build or run time. Binaries built with IPv6 support work well in both IPv4-only and dual family environments.
* Built-in features offer mechanisms to tune which links you wish to follow (see Following Links).
* The progress of individual downloads is traced using a progress gauge. Interactive downloads are tracked using a “thermometer”-style gauge, whereas non-interactive ones are traced with dots, each dot representing a fixed amount of data received (1KB by default). Either gauge can be customized to your preferences.
* Most of the features are fully configurable, either through command line options, or via the initialization file .wgetrc (see Startup File). Wget allows you to define global startup files (/usr/local/etc/wgetrc by default) for site settings. You can also specify the location of a startup file with the –config option.
* Finally, GNU Wget is free software. This means that everyone may use it, redistribute it and/or modify it under the terms of the GNU General Public License, as published by the Free Software Foundation (see the file COPYING that came with GNU Wget, for details).

As of version 1.14, Wget supports WARC output. See http://www.archiveteam.org/index.php?title=Wget_with_WARC_output for details of the development of this feature.

== Platform ==

GNU Wget can be installed on Unix-like systems (UNIX, Linux), Mac OS, and Windows computers.

=== Installation ===

* Unix-like systems: Most package managers include Wget, but they may not include the latest version. To get a later version with support for WARC, for example, Linux and UNIX users should compile the latest version of the source code following the instructions at http://wget.addictivecode.org/FrequentlyAskedQuestions#How_do_I_compile_Wget.3F.

* Macintosh: The default Mac OS does not include Wget. Source code can be compiled for Mac OS X or users can install an alternative package manager such as Homebrew (Homebrew installs the latest version by default). See http://coolestguidesontheplanet.com/install-and-configure-wget-on-os-x/ for instructions on how to install from source.

* Windows: packages for later versions of Wget compiled for Windows are available at http://eternallybored.org/misc/wget/.

==Documentation==
The user manual is available at http://www.gnu.org/software/wget/manual/wget.html. The manual is also available via man wget in Unix-like systems.

Additional documentation, including an FAQ, is available on the Wget wiki, http://wget.addictivecode.org/Wget.

= User Experiences =

* Milligan, Ian. (2012). Automated downloading with Wget. http://programminghistorian.org/lessons/automated-downloading-with-wget
* ArchiveTeam. (2014). Wget. http://www.archiveteam.org/index.php?title=Wget

= Development Activity =

{{Infobox_tool_details
|ohloh_id=Wget
}}

User:Danielle plumer

2014-10-01T02:22:02Z

Danielle plumer:

== About Me ==
I am a digital collections consultant working with cultural heritage institutions interested in putting their collections online. I work primarily in the areas of project planning and metadata creation, standards, and normalization. I also do research into information extraction from textual materials and natural language processing for information retrieval.

From 2005-2011, I coordinated the Texas Heritage Online program at the Texas State Library and Archives Commission, where I consulted with and assisted participants from the library, archives, and museum communities who were developing digital library projects.

I also teach graduate-level courses for the College of Information at the University of North Texas and for the School of Information at The University of Texas at Austin, including courses on metadata, digitization, and digital preservation. In addition to these formal courses, I developed and and so-taught a series of workshops offered throughout Texas as part of a grant funded by the Institute for Museum and Library Services. These workshops covered Digital Project Planning and Management Basics, Digital Archives Systems and Applications, Metadata Standards and Crosswalks, and Principles of Controlled Vocabulary and Thesaurus Design as well as supplemental courses on Digital Preservation Planning and Management and Digital Preservation Tools.

I earned an M.S. in Information Studies at The University of Texas at Austin in 2003. Prior to that, I earned a Ph.D. in English at the University of California, Davis.

==== Personal Links: ====
* [[Help:Editing | Guidelines for Editing COPTR]]
* [https://www.mediawiki.org/wiki/Help:Editing MediaWiki Editing Help]

==== Notes: ====
* To add an image to the InfoBox, first upload an image. For best results, images should be approx. 200 px. wide. PNG and JPG are both verified to work.
** Use the syntax |image=Gnu2.png within the InfoBox.
** Example: http://coptr.digipres.org/index.php?title=GNU_Wget

== Conflict of Interest ==
I am an independent consultant. My business, [http://www.dcplumer.com dcplumer associates], works with various libraries, archives, museums, and nonprofit organizations, and some of my paid work involves search engine optimization and use of social media (including Wikipedia) for digital collections. My goal as an editor is to advance the aims of COPTR. If other editors feel that my editing behavior is not advancing the aims of COPTR or that I am not following the [[Help:Editing | Guidelines for Editing COPTR]], I will be happy to modify my editing behavior. Please do not hesitate to contact me, via my [[User:Danielle_plumer | user talk]] page, if you have questions about my behavior.

GNU Wget

2014-10-01T02:19:42Z

Danielle plumer:

{{Infobox_tool
|purpose= Non-interactive network downloader
|image=Gnu2.png
|homepage=http://www.gnu.org/software/wget/
|license=GNU General Public License
|platforms=Unix, Linux, Windows, Macintosh
}}


[[Category:Web Crawl]]

= Description =
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

== Features ==

From the Wget manual:

* Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.
* Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.” While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). * * Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.
* File name wildcard matching and recursive mirroring of directories are available when retrieving via FTP. Wget can read the time-stamp information given by both HTTP and FTP servers, and store it locally. Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. This makes Wget suitable for mirroring of FTP sites, as well as home pages.
* Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.
* Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. Wget uses the passive FTP downloading by default, active FTP being an option.
* Wget supports IP version 6, the next generation of IP. IPv6 is autodetected at compile-time, and can be disabled at either build or run time. Binaries built with IPv6 support work well in both IPv4-only and dual family environments.
* Built-in features offer mechanisms to tune which links you wish to follow (see Following Links).
* The progress of individual downloads is traced using a progress gauge. Interactive downloads are tracked using a “thermometer”-style gauge, whereas non-interactive ones are traced with dots, each dot representing a fixed amount of data received (1KB by default). Either gauge can be customized to your preferences.
* Most of the features are fully configurable, either through command line options, or via the initialization file .wgetrc (see Startup File). Wget allows you to define global startup files (/usr/local/etc/wgetrc by default) for site settings. You can also specify the location of a startup file with the –config option.
* Finally, GNU Wget is free software. This means that everyone may use it, redistribute it and/or modify it under the terms of the GNU General Public License, as published by the Free Software Foundation (see the file COPYING that came with GNU Wget, for details).

As of version 1.14, Wget supports WARC output. See http://www.archiveteam.org/index.php?title=Wget_with_WARC_output for details of the development of this feature.

== Platform ==

GNU Wget can be installed on Unix-like systems (UNIX, Linux), Mac OS, and Windows computers.

=== Installation ===

* Unix-like systems: Most package managers include Wget, but they may not include the latest version. To get a later version with support for WARC, for example, Linux and UNIX users should compile the latest version of the source code following the instructions at http://wget.addictivecode.org/FrequentlyAskedQuestions#How_do_I_compile_Wget.3F.

* Macintosh: The default Mac OS does not include Wget. Source code can be compiled for Mac OS X or users can install an alternative package manager such as Homebrew (it is unknown which version of Wget Homebrew installs). See http://coolestguidesontheplanet.com/install-and-configure-wget-on-os-x/ for instructions on how to install from source.

* Windows: packages for later versions of Wget compiled for Windows are available at http://eternallybored.org/misc/wget/.

==Documentation==
The user manual is available at http://www.gnu.org/software/wget/manual/wget.html. The manual is also available via man wget in Unix-like systems.

Additional documentation, including an FAQ, is available on the Wget wiki, http://wget.addictivecode.org/Wget.

= User Experiences =

* Milligan, Ian. (2012). Automated downloading with Wget. http://programminghistorian.org/lessons/automated-downloading-with-wget
* ArchiveTeam. (2014). Wget. http://www.archiveteam.org/index.php?title=Wget

= Development Activity =

{{Infobox_tool_details
|ohloh_id=Wget
}}

File:Gnu2.png

2014-10-01T02:18:30Z

Danielle plumer:

GNU Wget

2014-10-01T02:06:20Z

Danielle plumer:

{{Infobox_tool
|purpose= Non-interactive network downloader
|image=gnu_small.png
|homepage=http://www.gnu.org/software/wget/
|license=GNU General Public License
|platforms=Unix, Linux, Windows, Macintosh
}}


[[Category:Web Crawl]]

= Description =
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

== Features ==

From the Wget manual:

* Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.
* Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.” While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). * * Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.
* File name wildcard matching and recursive mirroring of directories are available when retrieving via FTP. Wget can read the time-stamp information given by both HTTP and FTP servers, and store it locally. Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. This makes Wget suitable for mirroring of FTP sites, as well as home pages.
* Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.
* Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. Wget uses the passive FTP downloading by default, active FTP being an option.
* Wget supports IP version 6, the next generation of IP. IPv6 is autodetected at compile-time, and can be disabled at either build or run time. Binaries built with IPv6 support work well in both IPv4-only and dual family environments.
* Built-in features offer mechanisms to tune which links you wish to follow (see Following Links).
* The progress of individual downloads is traced using a progress gauge. Interactive downloads are tracked using a “thermometer”-style gauge, whereas non-interactive ones are traced with dots, each dot representing a fixed amount of data received (1KB by default). Either gauge can be customized to your preferences.
* Most of the features are fully configurable, either through command line options, or via the initialization file .wgetrc (see Startup File). Wget allows you to define global startup files (/usr/local/etc/wgetrc by default) for site settings. You can also specify the location of a startup file with the –config option.
* Finally, GNU Wget is free software. This means that everyone may use it, redistribute it and/or modify it under the terms of the GNU General Public License, as published by the Free Software Foundation (see the file COPYING that came with GNU Wget, for details).

As of version 1.14, Wget supports WARC output. See http://www.archiveteam.org/index.php?title=Wget_with_WARC_output for details of the development of this feature.

== Platform ==

GNU Wget can be installed on Unix-like systems (UNIX, Linux), Mac OS, and Windows computers.

=== Installation ===

* Unix-like systems: Most package managers include Wget, but they may not include the latest version. To get a later version with support for WARC, for example, Linux and UNIX users should compile the latest version of the source code following the instructions at http://wget.addictivecode.org/FrequentlyAskedQuestions#How_do_I_compile_Wget.3F.

* Macintosh: The default Mac OS does not include Wget. Source code can be compiled for Mac OS X or users can install an alternative package manager such as Homebrew (it is unknown which version of Wget Homebrew installs). See http://coolestguidesontheplanet.com/install-and-configure-wget-on-os-x/ for instructions on how to install from source.

* Windows: packages for later versions of Wget compiled for Windows are available at http://eternallybored.org/misc/wget/.

==Documentation==
The user manual is available at http://www.gnu.org/software/wget/manual/wget.html. The manual is also available via man wget in Unix-like systems.

Additional documentation, including an FAQ, is available on the Wget wiki, http://wget.addictivecode.org/Wget.

= User Experiences =

* Milligan, Ian. (2012). Automated downloading with Wget. http://programminghistorian.org/lessons/automated-downloading-with-wget
* ArchiveTeam. (2014). Wget. http://www.archiveteam.org/index.php?title=Wget

= Development Activity =

{{Infobox_tool_details
|ohloh_id=Wget
}}

File:Gnu small.png

2014-10-01T02:05:12Z

Danielle plumer:

GNU Wget

2014-10-01T01:48:22Z

Danielle plumer:

{{Infobox_tool
|purpose= Non-interactive network downloader
|image=[[File:Gnu.jpg]]
|homepage=http://www.gnu.org/software/wget/
|license=GNU General Public License
|platforms=Unix, Linux, Windows, Macintosh
}}


[[Category:Web Crawl]]

= Description =
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

== Features ==

From the Wget manual:

* Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.
* Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.” While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). * * Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.
* File name wildcard matching and recursive mirroring of directories are available when retrieving via FTP. Wget can read the time-stamp information given by both HTTP and FTP servers, and store it locally. Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. This makes Wget suitable for mirroring of FTP sites, as well as home pages.
* Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.
* Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. Wget uses the passive FTP downloading by default, active FTP being an option.
* Wget supports IP version 6, the next generation of IP. IPv6 is autodetected at compile-time, and can be disabled at either build or run time. Binaries built with IPv6 support work well in both IPv4-only and dual family environments.
* Built-in features offer mechanisms to tune which links you wish to follow (see Following Links).
* The progress of individual downloads is traced using a progress gauge. Interactive downloads are tracked using a “thermometer”-style gauge, whereas non-interactive ones are traced with dots, each dot representing a fixed amount of data received (1KB by default). Either gauge can be customized to your preferences.
* Most of the features are fully configurable, either through command line options, or via the initialization file .wgetrc (see Startup File). Wget allows you to define global startup files (/usr/local/etc/wgetrc by default) for site settings. You can also specify the location of a startup file with the –config option.
* Finally, GNU Wget is free software. This means that everyone may use it, redistribute it and/or modify it under the terms of the GNU General Public License, as published by the Free Software Foundation (see the file COPYING that came with GNU Wget, for details).

As of version 1.14, Wget supports WARC output. See http://www.archiveteam.org/index.php?title=Wget_with_WARC_output for details of the development of this feature.

== Platform ==

GNU Wget can be installed on Unix-like systems (UNIX, Linux), Mac OS, and Windows computers.

=== Installation ===

* Unix-like systems: Most package managers include Wget, but they may not include the latest version. To get a later version with support for WARC, for example, Linux and UNIX users should compile the latest version of the source code following the instructions at http://wget.addictivecode.org/FrequentlyAskedQuestions#How_do_I_compile_Wget.3F.

* Macintosh: The default Mac OS does not include Wget. Source code can be compiled for Mac OS X or users can install an alternative package manager such as Homebrew (it is unknown which version of Wget Homebrew installs). See http://coolestguidesontheplanet.com/install-and-configure-wget-on-os-x/ for instructions on how to install from source.

* Windows: packages for later versions of Wget compiled for Windows are available at http://eternallybored.org/misc/wget/.

==Documentation==
The user manual is available at http://www.gnu.org/software/wget/manual/wget.html. The manual is also available via man wget in Unix-like systems.

Additional documentation, including an FAQ, is available on the Wget wiki, http://wget.addictivecode.org/Wget.

= User Experiences =

* Milligan, Ian. (2012). Automated downloading with Wget. http://programminghistorian.org/lessons/automated-downloading-with-wget
* ArchiveTeam. (2014). Wget. http://www.archiveteam.org/index.php?title=Wget

= Development Activity =

{{Infobox_tool_details
|ohloh_id=Wget
}}

File:Gnu.jpg

2014-10-01T01:47:32Z

Danielle plumer:

GNU Wget

2014-10-01T00:58:14Z

Danielle plumer:

{{Infobox_tool
|purpose= Non-interactive network downloader
|image=
|homepage=http://www.gnu.org/software/wget/
|license=GNU General Public License
|platforms=Unix, Linux, Windows, Macintosh
}}


[[Category:Web Crawl]]

= Description =
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

== Features ==

From the Wget manual:

* Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.
* Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.” While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). * * Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.
* File name wildcard matching and recursive mirroring of directories are available when retrieving via FTP. Wget can read the time-stamp information given by both HTTP and FTP servers, and store it locally. Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. This makes Wget suitable for mirroring of FTP sites, as well as home pages.
* Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.
* Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. Wget uses the passive FTP downloading by default, active FTP being an option.
* Wget supports IP version 6, the next generation of IP. IPv6 is autodetected at compile-time, and can be disabled at either build or run time. Binaries built with IPv6 support work well in both IPv4-only and dual family environments.
* Built-in features offer mechanisms to tune which links you wish to follow (see Following Links).
* The progress of individual downloads is traced using a progress gauge. Interactive downloads are tracked using a “thermometer”-style gauge, whereas non-interactive ones are traced with dots, each dot representing a fixed amount of data received (1KB by default). Either gauge can be customized to your preferences.
* Most of the features are fully configurable, either through command line options, or via the initialization file .wgetrc (see Startup File). Wget allows you to define global startup files (/usr/local/etc/wgetrc by default) for site settings. You can also specify the location of a startup file with the –config option.
* Finally, GNU Wget is free software. This means that everyone may use it, redistribute it and/or modify it under the terms of the GNU General Public License, as published by the Free Software Foundation (see the file COPYING that came with GNU Wget, for details).

As of version 1.14, Wget supports WARC output. See http://www.archiveteam.org/index.php?title=Wget_with_WARC_output for details of the development of this feature.

== Platform ==

GNU Wget can be installed on Unix-like systems (UNIX, Linux), Mac OS, and Windows computers.

=== Installation ===

* Unix-like systems: Most package managers include Wget, but they may not include the latest version. To get a later version with support for WARC, for example, Linux and UNIX users should compile the latest version of the source code following the instructions at http://wget.addictivecode.org/FrequentlyAskedQuestions#How_do_I_compile_Wget.3F.

* Macintosh: The default Mac OS does not include Wget. Source code can be compiled for Mac OS X or users can install an alternative package manager such as Homebrew (it is unknown which version of Wget Homebrew installs). See http://coolestguidesontheplanet.com/install-and-configure-wget-on-os-x/ for instructions on how to install from source.

* Windows: packages for later versions of Wget compiled for Windows are available at http://eternallybored.org/misc/wget/.

==Documentation==
The user manual is available at http://www.gnu.org/software/wget/manual/wget.html. The manual is also available via man wget in Unix-like systems.

Additional documentation, including an FAQ, is available on the Wget wiki, http://wget.addictivecode.org/Wget.

= User Experiences =

* Milligan, Ian. (2012). Automated downloading with Wget. http://programminghistorian.org/lessons/automated-downloading-with-wget
* ArchiveTeam. (2014). Wget. http://www.archiveteam.org/index.php?title=Wget

= Development Activity =

{{Infobox_tool_details
|ohloh_id=Wget
}}

GNU Wget

2014-10-01T00:31:43Z

Danielle plumer:

{{Infobox_tool
|purpose= Non-interactive network downloader
|image=
|homepage=http://www.gnu.org/software/wget/
|license=GNU General Public License
|platforms=Unix, Linux, Windows, Macintosh
}}


[[Category:Web Crawl]]

= Description =
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

== Features ==

From the Wget manual:

* Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.
* Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.” While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). * * Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.
* File name wildcard matching and recursive mirroring of directories are available when retrieving via FTP. Wget can read the time-stamp information given by both HTTP and FTP servers, and store it locally. Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. This makes Wget suitable for mirroring of FTP sites, as well as home pages.
* Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.
* Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. Wget uses the passive FTP downloading by default, active FTP being an option.
* Wget supports IP version 6, the next generation of IP. IPv6 is autodetected at compile-time, and can be disabled at either build or run time. Binaries built with IPv6 support work well in both IPv4-only and dual family environments.
* Built-in features offer mechanisms to tune which links you wish to follow (see Following Links).
* The progress of individual downloads is traced using a progress gauge. Interactive downloads are tracked using a “thermometer”-style gauge, whereas non-interactive ones are traced with dots, each dot representing a fixed amount of data received (1KB by default). Either gauge can be customized to your preferences.
* Most of the features are fully configurable, either through command line options, or via the initialization file .wgetrc (see Startup File). Wget allows you to define global startup files (/usr/local/etc/wgetrc by default) for site settings. You can also specify the location of a startup file with the –config option.
* Finally, GNU Wget is free software. This means that everyone may use it, redistribute it and/or modify it under the terms of the GNU General Public License, as published by the Free Software Foundation (see the file COPYING that came with GNU Wget, for details).

As of version 1.14, Wget supports WARC output. See http://www.archiveteam.org/index.php?title=Wget_with_WARC_output for details of the development of this feature.

==Documentation==
The user manual is available at http://www.gnu.org/software/wget/manual/wget.html. The manual is also available via man wget in Unix-like systems.

Additional documentation, including an FAQ, is available on the Wget wiki, http://wget.addictivecode.org/Wget.

== Installation ==

Unix-like systems: Most package managers include Wget, but they may not include the latest version. To get a later version with support for WARC, for example, Linux, Unix, and Macintosh users should compile the latest version of the source code following the instructions at http://wget.addictivecode.org/FrequentlyAskedQuestions#How_do_I_compile_Wget.3F.

Windows: packages for later versions of Wget compiled for Windows are available at http://eternallybored.org/misc/wget/.

= User Experiences =

* Milligan, Ian. (2012). Automated downloading with Wget. http://programminghistorian.org/lessons/automated-downloading-with-wget
* ArchiveTeam. (2014). Wget. http://www.archiveteam.org/index.php?title=Wget

= Development Activity =

{{Infobox_tool_details
|ohloh_id=Wget
}}

User:Danielle plumer

2014-09-30T23:36:47Z

Danielle plumer:

== About Me ==
I am a digital collections consultant working with cultural heritage institutions interested in putting their collections online. I work primarily in the areas of project planning and metadata creation, standards, and normalization. I also do research into information extraction from textual materials and natural language processing for information retrieval.

From 2005-2011, I coordinated the Texas Heritage Online program at the Texas State Library and Archives Commission, where I consulted with and assisted participants from the library, archives, and museum communities who were developing digital library projects.

I also teach graduate-level courses for the College of Information at the University of North Texas and for the School of Information at The University of Texas at Austin, including courses on metadata, digitization, and digital preservation. In addition to these formal courses, I developed and and so-taught a series of workshops offered throughout Texas as part of a grant funded by the Institute for Museum and Library Services. These workshops covered Digital Project Planning and Management Basics, Digital Archives Systems and Applications, Metadata Standards and Crosswalks, and Principles of Controlled Vocabulary and Thesaurus Design as well as supplemental courses on Digital Preservation Planning and Management and Digital Preservation Tools.

I earned an M.S. in Information Studies at The University of Texas at Austin in 2003. Prior to that, I earned a Ph.D. in English at the University of California, Davis.

==== Personal Links: ====
* [[Help:Editing | Guidelines for Editing COPTR]]
* [https://www.mediawiki.org/wiki/Help:Editing MediaWiki Editing Help]

== Conflict of Interest ==
I am an independent consultant. My business, [http://www.dcplumer.com dcplumer associates], works with various libraries, archives, museums, and nonprofit organizations, and some of my paid work involves search engine optimization and use of social media (including Wikipedia) for digital collections. My goal as an editor is to advance the aims of COPTR. If other editors feel that my editing behavior is not advancing the aims of COPTR or that I am not following the [[Help:Editing | Guidelines for Editing COPTR]], I will be happy to modify my editing behavior. Please do not hesitate to contact me, via my [[User:Danielle_plumer | user talk]] page, if you have questions about my behavior.

User:Danielle plumer

2014-09-30T23:35:29Z

Danielle plumer:

== About Me ==
I am a digital collections consultant working with cultural heritage institutions interested in putting their collections online. I work primarily in the areas of project planning and metadata creation, standards, and normalization. I also do research into information extraction from textual materials and natural language processing for information retrieval.

From 2005-2011, I coordinated the Texas Heritage Online program at the Texas State Library and Archives Commission, where I consulted with and assisted participants from the library, archives, and museum communities who were developing digital library projects.

I also teach graduate-level courses for the College of Information at the University of North Texas and for the School of Information at The University of Texas at Austin, including courses on metadata, digitization, and digital preservation. In addition to these formal courses, I developed and and so-taught a series of workshops offered throughout Texas as part of a grant funded by the Institute for Museum and Library Services. These workshops covered Digital Project Planning and Management Basics, Digital Archives Systems and Applications, Metadata Standards and Crosswalks, and Principles of Controlled Vocabulary and Thesaurus Design as well as supplemental courses on Digital Preservation Planning and Management and Digital Preservation Tools.

I earned an M.S. in Information Studies at The University of Texas at Austin in 2003. Prior to that, I earned a Ph.D. in English at the University of California, Davis.

==== Personal Links: ====
* [[Help:Editing | Guidelines for Editing COPTR]]
* [https://www.mediawiki.org/wiki/Help:Editing MediaWiki Editing Help]

== Conflict of Interest ==
I am an independent consultant. My business, [http://www.dcplumer.com dcplumer associates], works with various libraries, archives, museums, and nonprofit organizations, and some of my paid work involves search engine optimization and use of social media (including Wikipedia) for digital collections. My goal as an editor is to advance the aims of COPTR. If other editors feel that my editing behavior is not advancing the aims of COPTR or that I am not following the within the [[Help:Editing | Guidelines for Editing COPTR]], I will be happy to modify my editing behavior. Please do not hesitate to contact me, via my [http://coptr.digipres.org/User:Danielle_plumer user talk page, if you have questions about my behavior.

User:Danielle plumer

2014-09-30T23:34:37Z

Danielle plumer:

== About Me ==
I am a digital collections consultant working with cultural heritage institutions interested in putting their collections online. I work primarily in the areas of project planning and metadata creation, standards, and normalization. I also do research into information extraction from textual materials and natural language processing for information retrieval.

From 2005-2011, I coordinated the Texas Heritage Online program at the Texas State Library and Archives Commission, where I consulted with and assisted participants from the library, archives, and museum communities who were developing digital library projects.

I also teach graduate-level courses for the College of Information at the University of North Texas and for the School of Information at The University of Texas at Austin, including courses on metadata, digitization, and digital preservation. In addition to these formal courses, I developed and and so-taught a series of workshops offered throughout Texas as part of a grant funded by the Institute for Museum and Library Services. These workshops covered Digital Project Planning and Management Basics, Digital Archives Systems and Applications, Metadata Standards and Crosswalks, and Principles of Controlled Vocabulary and Thesaurus Design as well as supplemental courses on Digital Preservation Planning and Management and Digital Preservation Tools.

I earned an M.S. in Information Studies at The University of Texas at Austin in 2003. Prior to that, I earned a Ph.D. in English at the University of California, Davis.

==== Personal Links: ====
* [[Help:Editing | Guidelines for Editing COPTR]]
* [https://www.mediawiki.org/wiki/Help:Editing MediaWiki Editing Help]

== Conflict of Interest ==
I am an independent consultant. My business, [http://www.dcplumer.com dcplumer associates], works with various libraries, archives, museums, and nonprofit organizations, and some of my paid work involves search engine optimization and use of social media (including Wikipedia) for digital collections. My goal as an editor is to advance the aims of COPTR. If other editors feel that my editing behavior is not advancing the aims of COPTR or that I am not following the within the [[Help:Editing | Guidelines for Editing COPTR]], I will be happy to modify my editing behavior. Please do not hesitate to contact me, via my User:Danielle_plumer page, if you have questions about my behavior.

User:Danielle plumer

2014-09-30T23:32:10Z

Danielle plumer: Created page with "== About Me == I am a digital collections consultant working with cultural heritage institutions interested in putting their collections online. I work primarily in the areas ..."

== About Me ==
I am a digital collections consultant working with cultural heritage institutions interested in putting their collections online. I work primarily in the areas of project planning and metadata creation, standards, and normalization. I also do research into information extraction from textual materials and natural language processing for information retrieval.

From 2005-2011, I coordinated the Texas Heritage Online program at the Texas State Library and Archives Commission, where I consulted with and assisted participants from the library, archives, and museum communities who were developing digital library projects.

I also teach graduate-level courses for the College of Information at the University of North Texas and for the School of Information at The University of Texas at Austin, including courses on metadata, digitization, and digital preservation. In addition to these formal courses, I developed and and so-taught a series of workshops offered throughout Texas as part of a grant funded by the Institute for Museum and Library Services. These workshops covered Digital Project Planning and Management Basics, Digital Archives Systems and Applications, Metadata Standards and Crosswalks, and Principles of Controlled Vocabulary and Thesaurus Design as well as supplemental courses on Digital Preservation Planning and Management and Digital Preservation Tools.

I earned an M.S. in Information Studies at The University of Texas at Austin in 2003. Prior to that, I earned a Ph.D. in English at the University of California, Davis.

==== Personal Links: ====
* [[Help:Editing | Guidelines for Editing COPTR]]
* [https://www.mediawiki.org/wiki/Help:Editing MediaWiki Editing Help]

== Conflict of Interest ==
I am an independent consultant. My business, [http://www.dcplumer.com dcplumer associates], works with various libraries, archives, museums, and nonprofit organizations, and some of my paid work involves search engine optimization and use of social media (including Wikipedia) for digital collections. My goal as an editor is to advance the aims of COPTR. If other editors feel that my editing behavior is not advancing the aims of COPTR or that I am not following the within the [[Help:Editing | Guidelines for Editing COPTR]], I will be happy to modify my editing behavior. Please do not hesitate to contact me, via my [[User:Danielle_plumer | user talk]] page, if you have questions about my behavior.