mxdyeah/nginx-modules
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add ngx_http_uploadprogress_module

Browse Source
This commit is contained in:
mxd
2026-05-17 11:46:42 +08:00
parent 9f28fed00a
commit c631e86b9a
10 changed files with 2331 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
README.md
View File

@@ -155,4 +155,26 @@ This module is recommended to be built statically because it interacts closely w
### License
BSD 2-Clause License.
---
## ngx_http_uploadprogress_module
nginx_uploadprogress_module is an implementation of an upload progress system, that monitors
RFC1867 POST upload as they are transmitted to upstream servers.
### Build
```bash
./configure \
--add-dynamic-module=/path/to/ngx_http_uploadprogress_module
```
### Notes
when compiled with --with-debug, this module will produce high number of log messages.
### License
BSD 2-Clause License.

115
ngx_http_uploadprogress_module/CHANGES Normal file
View File

@@ -0,0 +1,115 @@
nginx_upload_progress release 0.9.3 29 Jun 2024
* Allow to build with recent nginx versions
nginx_upload_progress release 0.9.2 03 Aug 2016
* Allow to build as an externally loadable module (thanks to Peter Tonoli).
nginx_upload_progress release 0.9.0 06 Apr 2012
* INCOMPATIBLE CHANGE: JSONP is now the default output for nginx upload
progress module. To restore the old behavior, add:
upload_progress_java_output
to your nginx configuration (thanks to drewbuschhorn for this work).
* Add instructions in the README on how to use this module with JQuery
AJAX.
nginx_upload_progress release 0.8.4 24 Feb 2012
* Fix compatibility with nginx 1.1.15
nginx_upload_progress release 0.8.3 11 Sep 2011
* HTTP redirects (ie 3xx range) shouldn't be reported as an error (thanks to
Pierre-Yves Kerembellec)
* Fix a double remove in the rb tree due to an uninitialized flag (thanks to
Theo Cushion)
nginx_upload_progress release 0.8.2 07 Nov 2010
* Session ID query string parameter name is now configurable (thanks to
Pierre-Yves Kerembellec)
nginx_upload_progress release 0.8.1 27 Feb 2010
* Fixed find_node so that it can find correctly nodes when there is an
hash collision (thanks to Markus Doppelbauer for his detailed bug report
and test case).
nginx_upload_progress release 0.8 19 Dec 2009
* fixed segfault at start on some platforms (reported by Vladimir
Getmanshchuk and Denis Denisenko)
* implemented JSONP output (thanks to Bruno Deferrari)
* X-Progress-ID can now be the last parameter in the request (thanks to
Bruno Deferrari)
nginx_upload_progress release 0.7 21 Nov 2009
* fixed segfault when uploads are aborted (thanks to Markus Doppelbauer for
his bug report)
* session ID header name is now configurable (thanks to Valery Kholodkov)
* Added directive to format output as pure json (thanks to Valery Kholodkov)
* Added directive to format output with configurable template (thanks to
Valery Kholodkov)
* Added directive to set a probe response content-type (thanks to Valery
Kholodkov)
* Added upload status variables (needs a status patch) (thanks to Valery
Kholodkov)
nginx_upload_progress release 0.6 12 Sep 2009
* fixed node corruption when cleaning uploads (thanks to Vladimir
Getmanshchuk for his detailed report)
nginx_upload_progress release 0.5 28 Jun 2008
* fixed progress probes returning "done" when upload was still in progress
(thanks to Alexandr Kutuzov for his detailed report)
* fixed stuck worker process because of rescheduled timer when issuing a
reload or graceful quit (thanks to Alexandr Kutuzov for his detailed
report)
nginx_upload_progress release 0.4 18 May 2008
* Incompatible change: track_uploads MUST be the last directive of a location
it also must be in a proxy_pass or fastcgi_pass location.
* fixed multiple worker_process issue
* fixed several crash conditions
nginx_upload_progress release 0.3 06 May 2008
* fixed crash if the upload was denied by nginx because of any error
condition (thanks to Michal Drapiewski for his detailed report)
* report original upload error condition to client in the upload progress
probe
* case-insensitive checking of the X-Progress-ID header to overcome Internet
Explorer XMLHttpRequest issue.
nginx_upload_progress release 0.2 10 Oct 2007
* the system now remembers old active uploads for 1 minute to be
able to send back either error status or done status to upload
progress probes.
* track of HTTP error 413 (request entity too large) is implemented
and the error status '413' is returned.
nginx_upload_progress release 0.1 3 Oct 2007
* public release

25
ngx_http_uploadprogress_module/LICENSE Normal file
View File

@@ -0,0 +1,25 @@
/*
* Copyright (C) 2007 Brice FIGUREAU
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
*/

8
ngx_http_uploadprogress_module/Makefile Normal file
View File

@@ -0,0 +1,8 @@
dist: CHANGES README LICENSE config ngx_http_uploadprogress_module.c
tar czvf ../nginx_uploadprogress_module-0.7.tar.gz \
../nginx_uploadprogress_module/CHANGES \
../nginx_uploadprogress_module/README \
../nginx_uploadprogress_module/LICENSE \
../nginx_uploadprogress_module/config \
../nginx_uploadprogress_module/ngx_http_uploadprogress_module.c

329
ngx_http_uploadprogress_module/README Normal file
View File

@@ -0,0 +1,329 @@
Nginx Upload Progress Module
============================
Introduction
============
nginx_uploadprogress_module is an implementation of an upload progress system, that monitors
RFC1867 POST upload as they are transmitted to upstream servers.
It works by tracking the uploads proxied by Nginx to upstream servers without
analysing the uploaded content and offers a web API to report upload progress in Javscript, Json or any
other format (through the help of templates).
It works because Nginx acts as an accelerator of an upstream server, storing uploaded POST content
on disk, before transmitting it to the upstream server. Each individual POST upload request
should contain a progress unique identifier.
This module is Copyright (c) 2007-2012 Brice Figureau, and is licensed under the BSD license (see LICENSE).
* rbtree and shm_zone code is based on Igor Sysoev limit_zone Nginx module.
* expire header code is based on Igor Sysoev header_filter Nginx module.
The JSON idea and the mechanism idea are based on Lighttpd mod_uploadprogress:
http://blog.lighttpd.net/articles/2006/08/01/mod_uploadprogress-is-back
WARNING:
* when compiled with --with-debug, this module will produce high number of log messages.
INCOMPATIBLE CHANGES
====================
v0.9.0:
JSONP is now the default output of the progress probes. If you rely on this module serving
the deprecated java output use:
upload_progress_java_output
in the progress probe location.
Installation
============
nginx_uploadprogress_module has been tested with Nginx 0.6.x, 0.7.x, 0.8.x and 1.0.x.
Download the Nginx sources from http://nginx.net/ and unpack it.
To build Nginx, change to the directory which contains the Nginx
sources, and run the configuration script making sure to add the path
to the nginx_uploadprogress_module sources using the --add-module option: ::
$ ./configure --add-module=/path/to/nginx_uploadprogress_module/
Now you can build and install the software:
$ make
and as root:
$ make install
Configuration
=============
Each upload request should be assigned a unique identifier. This unique identifier will be used
to store the request and reference it to report.
This identifier can be transmitted either as a GET argument or as an HTTP header whose name is X-Progress-ID.
upload_progress
+++++++++++++++
:Syntax: upload_progress <zone_name> <zone_size>
:Default: none
:Context: http
:Description:
This directive enables the upload progress module and reserve <zone_size> bytes to the <zone_name> which
will be used to store the per-connection tracking information.
track_uploads
+++++++++++++
:Syntax: track_uploads <zone_name> <timeout>
:Default: none
:Context: location
:Description:
This directive enables tracking uploads for the current location. Each POST landing in this location will register
the request in the <zone_name> upload progress tracker.
Since Nginx doesn't support yet RFC 1867 upload, the location must be a proxy_pass or fastcgi location.
The POST _must_ have a query parameter called X-Progress-ID (or an HTTP header of the same name) whose value is the
unique identifier used to get progress information. If the POST has no such information, the upload will not be tracked.
The tracked connections are kept at most <timeout> seconds after they have been finished to be able to serve
useful information to upload progress probes.
WARNING: this directive must be the last directive of the location. It must be in a proxy_pass or
fastcgi_pass location.
report_uploads
++++++++++++++
:Syntax: report_uploads <zone_name>
:Default: none
:Context: location
:Description:
This directive allows a location to report the upload progress that is tracked by track_uploads for <zone_name>.
The returned document is a Javascript text with the possible 4 results by default:
* the upload request hasn't been registered yet or is unknown:
new Object({ 'state' : 'starting' })
* the upload request has ended:
new Object({ 'state' : 'done' })
* the upload request generated an HTTP error
new Object({ 'state' : 'error', 'status' : <error code> })
one error code that can be of use to track for the client is 413 (request entity too large).
* the upload request is in progress:
new Object({ 'state' : 'uploading', 'received' : <size_received>, 'size' : <total_size>})
It is possible to return pure json instead of this javascript (see upload_progress_json_output).
It is also possible to configure completely the response format with the directive:
upload_progress_template
The HTTP request to this location must have a X-Progress-ID parameter or HTTP header containing a valid
unique identifier of an in progress upload.
upload_progress_content_type
++++++++++++++++++++++++++++
:Syntax: upload_progress_content_type <content_type>
:Default: text/javascript
:Context: location
:Description:
This directive allows to change the upload progress probe response content-type.
upload_progress_header
++++++++++++++++++++++
:Syntax: upload_progress_header <progress-id>
:Default: X-Progress-ID
:Context: location
:Description:
This directive allows to change the header name of the progress ID.
upload_progress_jsonp_parameter
++++++++++++++++++++++
:Syntax: upload_progress_jsonp_parameter <callback_parameter>
:Default: callback
:Context: location
:Description:
This directive allows to change the name of the GET parameter with the jsonp callback name.
upload_progress_java_output
+++++++++++++++++++++++++++
:Syntax: upload_progress_java_output
:Default: N/A
:Context: location
:Description:
This directive sets everything to output as eval() javascript compatible code.
upload_progress_json_output
+++++++++++++++++++++++++++
:Syntax: upload_progress_json_output
:Default: N/A
:Context: location
:Description:
This directive sets everything to output as pure json.
upload_progress_jsonp_output
++++++++++++++++++++++++++++
:Syntax: upload_progress_jsonp_output
:Default: N/A
:Context: location
:Description:
This directive sets everything to output as jsonp (like json output, but with callback).
upload_progress_template
++++++++++++++++++++++++
:Syntax: upload_progress_template <state> <template>
:Default: none
:Context: location
:Description:
This directive can be used to install a progress response template.
The available list of state is:
* starting
* uploading
* error
* done
Nginx will replace the value of the following variables with their respective
value for the upload:
* $uploadprogress_length: total size of the upload
* $uploadprogress_received: what the server has received so far
* $uploadprogress_status: error code in case of HTTP error
* $uploadprogress_callback: jsonp callback name if provided as a GET query parameter with name 'callback'
For instance to return XML (instead of the default Javascript or json):
upload_progress_content_type 'text/xml';
upload_progress_template starting '<upload><state>starting</state></upload>';
upload_progress_template uploading '<upload><state>uploading</state><size>$uploadprogress_length</size><uploaded>$uploadprogress_received</uploaded></upload>';
upload_progress_template done '<upload><state>done</state></upload>';
upload_progress_template error '<upload><state>error</state><code>$uploadprogress_status</code></upload>';
Example of jsonp response:
upload_progress_template starting "$uploadprogress_callback({ \"state\" : \"starting\"});";
upload_progress_template error "$uploadprogress_callback({ \"state\" : \"error\", \"status\" : $uploadprogress_status });";
upload_progress_template done "$uploadprogress_callback({ \"state\" : \"done\"});";
upload_progress_template uploading "$uploadprogress_callback({ \"state\" : \"uploading\", \"received\" : $uploadprogress_received, \"size\" : $uploadprogress_length });";
Configuration Example:
+++++++++++++++++++++
http {
# reserve 1MB under the name 'proxied' to track uploads
upload_progress proxied 1m;
server {
listen 127.0.0.1 default;
server_name _ *;
root /path/to/root;
location / {
# proxy to upstream server
proxy_pass http://127.0.0.1;
proxy_redirect default;
# track uploads in the 'proxied' zone
# remember connections for 30s after they finished
track_uploads proxied 30s;
}
location ^~ /progress {
# report uploads tracked in the 'proxied' zone
report_uploads proxied;
}
}
Usage Example
=============
(based on Lighttd mod_uploadprogress module example):
First we need a upload form:
<form id="upload" enctype="multipart/form-data"
action="/upload.php" method="post"
onsubmit="openProgressBar(); return true;">
<input type="hidden" name="MAX_FILE_SIZE" value="30000000" />
<input name="userfile" type="file" label="fileupload" />
<input type="submit" value="Send File" />
</form>
And a progress bar to visualize the progress:
<div>
<div id="progress" style="width: 400px; border: 1px solid black">
<div id="progressbar"
style="width: 1px; background-color: black; border: 1px solid white">
&nbsp;
</div>
</div>
<div id="tp">(progress)</div>
</div>
Then we need to generate the Unique Identifier and launch the upload on submit
action. This also will start the ajax progress report mechanism.
interval = null;
function openProgressBar() {
/* generate random progress-id */
uuid = "";
for (i = 0; i < 32; i++) {
uuid += Math.floor(Math.random() * 16).toString(16);
}
/* patch the form-action tag to include the progress-id */
document.getElementById("upload").action="/upload.php?X-Progress-ID=" + uuid;
/* call the progress-updater every 1000ms */
interval = window.setInterval(
function () {
fetch(uuid);
},
1000
);
}
function fetch(uuid) {
req = new XMLHttpRequest();
req.open("GET", "/progress", 1);
req.setRequestHeader("X-Progress-ID", uuid);
req.onreadystatechange = function () {
if (req.readyState == 4) {
if (req.status == 200) {
/* poor-man JSON parser */
var upload = eval(req.responseText);
document.getElementById('tp').innerHTML = upload.state;
/* change the width if the inner progress-bar */
if (upload.state == 'done' || upload.state == 'uploading') {
bar = document.getElementById('progressbar');
w = 400 * upload.received / upload.size;
bar.style.width = w + 'px';
}
/* we are done, stop the interval */
if (upload.state == 'done') {
window.clearTimeout(interval);
}
}
}
}
req.send(null);
}
Companion Software
==================
This software can also work with Valery Kholodkov' Nginx Upload Module:
http://www.grid.net.ru/nginx/upload.en.html
You can also use the following javascript libraries client side:
http://drogomir.com/blog/2008/6/30/upload-progress-script-with-safari-support
Note that when using jQuery AJAX for progress monitoring, such as:
https://github.com/drogus/jquery-upload-progress
you should be sure to set a upload_progress template parameter:
upload_progress_json_output
or
upload_progress_jsonp_output
depending on your jQuery AJAX dataType setting.

11
ngx_http_uploadprogress_module/config Normal file
View File

@@ -0,0 +1,11 @@
ngx_addon_name=ngx_http_uploadprogress_module
if test -n "$ngx_module_link"; then
ngx_module_type=HTTP_FILTER
ngx_module_name=ngx_http_uploadprogress_module
ngx_module_srcs="$ngx_addon_dir/ngx_http_uploadprogress_module.c"
. auto/module
else
HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES ngx_http_uploadprogress_module"
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_uploadprogress_module.c"
fi

1772
ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
ngx_http_uploadprogress_module/test/100 Normal file
View File

Binary file not shown.

8
ngx_http_uploadprogress_module/test/client.sh Normal file
View File

@@ -0,0 +1,8 @@
#!/bin/sh
# usage: client.sh UPLOAD_ID PROGRESS_URL
cont=""
while [ "$cont" != "new Object({ 'state' : 'done' })" ]
do
cont=`curl -s -H "x-progress-id: $1" $2 | sed -e 's/[\n\r]//'`
echo "[$1] '$cont'"
done

41
ngx_http_uploadprogress_module/test/stress.sh Normal file
View File

@@ -0,0 +1,41 @@
#!/bin/sh
# Usage: stress.sh UPLOAD_URL PROGRESS_URL
#
i=0
LIMIT="10k"
FILE="100"
#trap 'kill_all' SIGINT SIGTERM
while [ "1" == "1" ]
do
for j in $(seq 5)
do
i=`expr $i + 1`
echo "Upload $i"
curl --limit-rate $LIMIT -F pouet=@$FILE $1?X-Progress-ID=$i &
sh client.sh $i $2 &
i=`expr $i + 1`
echo "Upload $i"
curl --limit-rate $LIMIT -F pouet=@$FILE $1?X-Progress-ID=$i &
sh client.sh $i $2 &
i=`expr $i + 1`
echo "Upload $i"
curl --limit-rate $LIMIT -F pouet=@$FILE $1?X-Progress-ID=$i &
sh client.sh $i $2 &
i=`expr $i + 1`
echo "Upload $i"
curl --limit-rate $LIMIT -F pouet=@$FILE $1?X-Progress-ID=$i &
sh client.sh $i $2 &
i=`expr $i + 1`
echo "Upload $i"
curl --limit-rate $LIMIT -F pouet=@$FILE $1?X-Progress-ID=$i &
sh client.sh $i $2 &
i=`expr $i + 1`
echo "Upload $i"
curl --limit-rate $LIMIT -F pouet=@$FILE $1?X-Progress-ID=$i &
sh client.sh $i $2 &
done
wait
done