Lua-cURL

Author: Jürgen Hötzel
Contact: http://www.hoetzel.info/
Date: 2007-09-21
Copyright: This document has been placed in the public domain.

Contents

Introduction

This project is not a fork of LuaCURL, which is a direct mapping of parts of the libcurl-easy interface.

The intent of Lua-cURL is to adapt the

of libcurl to the functionality of Lua (for example by using iterators instead of callbacks when possible).

Installation

The Autotools chain is used to configure, build and install. Just invoke:

./configure && make install

If your Lua installation doesn't contain pkg-config support, you have to specify compiler/linker flags and the target directory:

LUA_CFLAGS="-I/usr/local/include" LUA_LIBS="-L/usr/local/lib -lm -llua" ./configure  --with-cmoddir=/usr/local/lib/lua/5.1

Easy interface

cURL.easy_init()
returns a new easy handle.
cURL.version_info()
returns a table containing version info and features/protocols sub table
easy:escape(string)
return URL encoded string
easy:unescape(string)
return URL decoded string
easy:setopt*(value)
libcurl properties an options are mapped to individual functions:
  • easy:setopt_url(string)
  • easy:setopt_verbose(number)
  • easy:setopt_proxytype(string)
  • easy:setopt_share(share) (See: share)
  • ...
easy:perform(table)
Perform the transfer as described in the options, using an optional callback table.The callback table indices are named after the equivalent cURL callbacks:
  • writefunction = function(str)
  • readfunction = function()
  • headerfunction = function(str)
easy:post(table)

Prepare a multipart/formdata post. The table indices are named after the form fields and should map to string values:

{field1 = value1,
 field2 = value1}

or more generic descriptions in tables:

{field1 = {file="/tmp/test.txt",
           type="text/plain"},
{field2 = {file="dummy.html",
           data="<html><bold>bold</bold></html>,
           type="text/html"}}

Example 1: Fetch the example.com homepage

require("cURL")

-- open output file
f = io.open("example_homepage", "w")

c = cURL.easy_init()
-- setup url
c:setopt_url("http://www.example.com/")
-- perform, invokes callbacks
c:perform({writefunction = function(str) 
				f:write(str)
			     end})

-- close output file
f:close()
print("Done")

Example 2: "On-The-Fly" file upload

-- simple "On the fly" fileupload

require("cURL")

c=cURL.easy_init()
c:setopt_url("ftp://ftptest:secret0815@targethost/file.dat")
c:setopt_upload(1)
count=0
c:perform({readfunction=function(n) 
			   count = count + 1
			   if (count < 10)  then
			      return "Line " .. count .. "\n"
			   end
			   return nil
			end})
print("Fileupload done")

Example 3: "Posting" data

require("cURL")

c = cURL.easy_init()

c:setopt_url("http://localhost")
postdata = {  
   -- post file from filesystem
   name = {file="post.lua",
	   type="text/plain"},
   -- post file from data variable
   name2 = {file="dummy.html",
	    data="<html><bold>bold</bold></html>",
	    type="text/html"}}
c:post(postdata)
c:perform()

print("Done")

Multi interface

cURL.multi_init()
returns a new multi handle
multi:add_handle(easy)
add an easy handle to a multi session
multi:perform()

returns an iterator function that, each time it is called, returns the next data, type and corresponding easy handle:

data:
data returned by the cURL library
type
type of data returned ("header" or "data")
easy
corresponding easy handle of the data returned

Example 1: "On-The-Fly" XML parsing

-- use LuaExpat and Lua-CuRL together for On-The-Fly XML parsing
require("lxp")
require("cURL")

tags = {}			
items = {}

callback = {}

function callback.StartElement(parser, tagname)
   tags[#tags + 1] = tagname
   if (tagname == "item") then
      items[#items + 1] = {}
   end
end

function callback.CharacterData(parser, str) 
   if (tags[#tags -1] == "item") then
      --we are parsing a item, get rid of trailing whitespace
      items[#items][tags[#tags]] = string.gsub(str, "%s*$", "")
   end
end
function callback.EndElement(parser, tagname)
   --assuming well formed xml
   tags[#tags] = nil		
end

p = lxp.new(callback)

-- create and setup easy handle
c = cURL.easy_init()
c:setopt_url("http://www.lua.org/news.rss")

m = cURL.multi_init()
m:add_handle(c)

for data,type in m:perform() do 
   -- ign "header"
   if (type == "data") then 
      assert(p:parse(data))
   end
end

--finish document
assert(p:parse())		
p:close()

for i, item in ipairs(items) do 
   for k, v in pairs(item) do 
      print(k,v)
   end
   print()		
end

Share interface

cURL.share_init()
returns a new share handle
share:setopt_share(string)
specifies the type of data that should be shared ("DNS" or "COOKIE")

Since Lua is single-threaded, there is no mapping for the lock options.

Example 1: Share Cookie date across easy handles

-- Cookie data will be shared across the easy handles to do an authorized download
require("cURL")

-- create share handle (share COOKIE and DNS Cache)
s = cURL.share_init()
s:setopt_share("COOKIE")
s:setopt_share("DNS")

-- create first easy handle to do the login
c = cURL.easy_init()
c:setopt_share(s)
c:setopt_url("http://targethost/login.php?username=foo&password=bar")

-- create second easy handle to do the download
c2 = cURL.easy_init()
c2:setopt_share(s)
c2:setopt_url("http://targethost/download.php?id=test")

--login
c:perform()

--download			
c2:perform()

Appendix

Using SSL

The cert bundle distributed with cURL may be out of date and cannot validate many certificates. You can supply a different PEM cert bundle by using easy:setopt_cainfo(string). I wrote a shell script (download) to convert the cacert keystore distributed with the Java Runtime Environment to PEM.