noury/mirror-yaf
1
0
Fork 0
mirror of https://github.com/lyx0/yaf.git synced 2024-11-13 19:49:53 +01:00
Code Issues Projects Releases Packages Wiki Activity

feat: add option to scrub EXIF tags on image files

Browse source 
EXIF scrubbing can be enabled via the `ScrubExif` config key. When
enabled, all standard EXIF tags (as defined by the IFD mappings in the
go-exif library) are removed on uploaded JPEG and PNG images.

The `ExifAllowedIds` and `ExifAllowedPaths` config keys can be used to
selectively allow specific tags to survive the scrubbing. This can be
useful if you want to preserve image orientation information for
example. The IDs for standard tags can be found in [1].

The path specification for `ExifAllowedPaths` relies on the format
implemented in go-exif which is "documented" in machine-readable format
in [2]. Multiple paths can be specified, separated by a space. The
path format is as follows:

    1. For tags in the main section: `IFD/<GroupName>/<FieldName>`.
       Examples: `IFD/Orientation`, `IFD/Exif/Flash`,
       `IFD/GPSInfo/GPSTimeStamp`. You will probably want to use both
       [1] and [2] in combination if you plan to specify allowed tags by
       path.
    2. Tags in the thumbnail section follow the same format but paths
       start with `IFD1/` instead of `IFD`.

[1]: https://exiv2.org/tags.html
[2]: a6301f85c8/assets/tags.yaml
This commit is contained in:
Leon Richardt 2022-08-16 00:23:29 +02:00
parent 445540c747
commit e0afb453a5
No known key found for this signature in database
GPG key ID: AD8BDD6273FE8FC5
12 changed files with 662 additions and 46 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

50
README.md
View file

@ -1,5 +1,5 @@
# jaf - Just Another Fileshare
jaf is a simple, zero-dependency Go program to handle file uploads.
jaf is a simple Go program to handle file uploads.
If you also want to serve the uploaded files, consider a web server like [nginx](https://nginx.org/en/).
## Installation
@ -28,19 +28,51 @@ Port: 4711
LinkPrefix: https://jaf.example.com/
FileDir: /var/www/jaf/
LinkLength: 5
ScrubExif: true
# Both IDs also refer to the "Orientation" tag, included for illustrative purposes only
ExifAllowedIds: 0x0112 274
ExifAllowedPaths: IFD/Orientation
ExifAbortOnError: true
```
Option | Use
------------ | -------------------------------------------------------------------
`Port` | the port number jaf will listen on
`LinkPrefix` | a string that will be prepended to the file name generated by jaf
`FileDir` | path to the directory jaf will save uploaded files in
`LinkLength` | the number of characters the generated file name is allowed to have
Option | Use
------------------ | -------------------------------------------------------------------
`Port` | the port number jaf will listen on
`LinkPrefix` | a string that will be prepended to the file name generated by jaf
`FileDir` | path to the directory jaf will save uploaded files in
`LinkLength` | the number of characters the generated file name is allowed to have
`ScrubExif` | whether to remove EXIF tags from uploaded JPEG and PNG images (`true` or `false`)
`ExifAllowedIds` | a space-separated list of EXIF tag IDs that should be preserved through EXIF scrubbing (only relevant if `ScrubExif` is `true`)
`ExifAllowedPaths` | a space-separated list of EXIF tag paths that should be preserved through EXIF scrubbing (only relevant if `ScrubExif` is `true`)
`ExifAbortOnError` | whether to abort JPEG and PNG uploads if an error occurs during EXIF scrubbing (only relevant if `ScrubExif` is `true`)
Make sure the user running jaf has suitable permissions to read, and write to, `FileDir`.
Also note that `LinkLength` directly relates to the number of files that can be saved.
Since jaf only uses alphanumeric characters for file name generation, a maximum of `(26 + 26 + 10)^LinkLength` names can be generated.
#### A Note on EXIF Scrubbing
EXIF scrubbing can be enabled via the `ScrubExif` config key.
When enabled, all standard EXIF tags are removed on uploaded JPEG and PNG images per default.
It is meant as a last-line "defense mechanism" against leaking PII, such as GPS information on pictures.
**If possible, you should always prefer disabling capturing potentially sensitive EXIF tags when creating the images!**
Obviously, EXIF tags serve a purpose and you may want to keep _some_ of the information, e.g., image orientation.
The `ExifAllowedIds` and `ExifAllowedPaths` config keys can be used to selectively allow specific tags to survive the scrubbing.
The IDs for standard tags can be found in [1].
You may specify tag IDs in decimal and hexadecimal notation.
(In the latter case, the ID _must_ start with `0x`.)
The path specification for `ExifAllowedPaths` relies on the format implemented in [`go-exif`](https://github.com/dsoprea/go-exif) which is "documented" in machine-readable format in [2].
Multiple paths can be specified, separated by a space.
The path format is as follows:
1. For tags in the main section: `IFD/<GroupName>/<FieldName>`.
Examples: `IFD/Orientation`, `IFD/Exif/Flash`, `IFD/GPSInfo/GPSTimeStamp`.
You will probably want to use both [1] and [2] in combination if you plan to specify allowed tags by path.
2. Tags in the thumbnail section follow the same format but paths start with `IFD1/` instead of `IFD`.
### nginx
If you use a reverse-proxy to forward requests to jaf, make sure to correctly forward the original request headers.
For nginx, this is achieved via the `proxy_pass_request_headers on;` option.
@ -90,3 +122,7 @@ Note that you may have to add additional header fields to the request, e.g. if y
## Inspiration
- [i](https://github.com/fourtf/i) by [fourtf](https://github.com/fourtf) – a project very similar in scope and size
- [filehost](https://github.com/nuuls/filehost) by [nuuls](https://github.com/nuuls) – a more integrated, fully-fledged solution that offers a web interface and also serves the files
[1]: https://exiv2.org/tags.html
[2]: https://github.com/dsoprea/go-exif/blob/a6301f85c82b0de82ceb8501f3c4a73ea7df01c2/assets/tags.yaml

89
config.go
View file

@ -13,10 +13,14 @@ const (
)
type Config struct {
Port int
LinkPrefix string
FileDir string
LinkLength int
Port int
LinkPrefix string
FileDir string
LinkLength int
ScrubExif bool
ExifAllowedIds []uint16
ExifAllowedPaths []string
ExifAbortOnError bool
}
func ConfigFromFile(filePath string) (*Config, error) {
@ -32,10 +36,14 @@ func ConfigFromFile(filePath string) (*Config, error) {
log.SetPrefix("config.FromFile > ")
retval := &Config{
Port: 4711,
LinkPrefix: "https://jaf.example.com/",
FileDir: "/var/www/jaf/",
LinkLength: 5,
Port: 4711,
LinkPrefix: "https://jaf.example.com/",
FileDir: "/var/www/jaf/",
LinkLength: 5,
ScrubExif: true,
ExifAllowedIds: []uint16{},
ExifAllowedPaths: []string{},
ExifAbortOnError: true,
}
scanner := bufio.NewScanner(file)
@ -47,13 +55,15 @@ func ConfigFromFile(filePath string) (*Config, error) {
continue
}
tokens := strings.Split(line, ": ")
if len(tokens) != 2 {
key, val, found := strings.Cut(line, ":")
if !found {
log.Printf("unexpected line: \"%s\", ignoring\n", line)
continue
}
key, val := strings.TrimSpace(tokens[0]), strings.TrimSpace(tokens[1])
key = strings.TrimSpace(key)
val = strings.TrimSpace(val)
switch key {
case "Port":
@ -74,6 +84,63 @@ func ConfigFromFile(filePath string) (*Config, error) {
}
retval.LinkLength = parsed
case "ScrubExif":
parsed, err := strconv.ParseBool(val)
if err != nil {
return nil, err
}
retval.ScrubExif = parsed
case "ExifAllowedIds":
if val == "" {
// No IDs specified at all
break
}
stringIds := strings.Split(val, " ")
parsedIds := make([]uint16, 0, len(stringIds))
for _, stringId := range stringIds {
var parsed uint64
var err error
if strings.HasPrefix(stringId, "0x") {
// Parse as a hexadecimal number
hexStringId := strings.Replace(stringId, "0x", "", 1)
parsed, err = strconv.ParseUint(hexStringId, 16, 16)
} else {
// Parse as a decimal number
parsed, err = strconv.ParseUint(stringId, 10, 16)
}
if err != nil {
log.Printf(
"Could not parse ID from: \"%s\", ignoring. Error: %s\n",
stringId,
err,
)
continue
}
parsedIds = append(parsedIds, uint16(parsed))
}
retval.ExifAllowedIds = parsedIds
case "ExifAllowedPaths":
if val == "" {
// No paths specified at all
break
}
paths := strings.Split(val, " ")
retval.ExifAllowedPaths = paths
case "ExifAbortOnError":
parsed, err := strconv.ParseBool(val)
if err != nil {
return nil, err
}
retval.ExifAbortOnError = parsed
default:
log.Printf("unexpected key: \"%s\", ignoring\n", key)
}

20
config_test.go
View file

@ -6,7 +6,21 @@ import (
func assertEqual[S comparable](have S, want S, t *testing.T) {
if have != want {
t.Error("have:", have, ", want:", want, "\n")
t.Error("have:", have, ", want:", want)
}
}
func assertEqualSlice[S comparable](have []S, want []S, t *testing.T) {
if len(have) != len(want) {
t.Error("lengths differ! have:", len(have), ", want:", len(want))
return
}
for i := range want {
if have[i] != want[i] {
t.Error("slices differ at position", i, ":", have[i], "!=", want[i])
return
}
}
}
@ -20,4 +34,8 @@ func TestConfigFromFile(t *testing.T) {
assertEqual(config.LinkPrefix, "https://jaf.example.com/", t)
assertEqual(config.FileDir, "/var/www/jaf/", t)
assertEqual(config.LinkLength, 5, t)
assertEqual(config.ScrubExif, true, t)
assertEqualSlice(config.ExifAllowedIds, []uint16{0x0112, 274}, t)
assertEqualSlice(config.ExifAllowedPaths, []string{"IFD/Orientation"}, t)
assertEqual(config.ExifAbortOnError, true, t)
}

5
example.conf
View file

@ -3,3 +3,8 @@ Port: 4711
LinkPrefix: https://jaf.example.com/
FileDir: /var/www/jaf/
LinkLength: 5
ScrubExif: true
# Both IDs also refer to the "Orientation" tag, included for illustrative purposes only
ExifAllowedIds: 0x0112 274
ExifAllowedPaths: IFD/Orientation
ExifAbortOnError: true

267
exifscrubber/exifscrubber.go Normal file
View file

@ -0,0 +1,267 @@
package exifscrubber
import (
"bytes"
"errors"
"fmt"
exif "github.com/dsoprea/go-exif/v3"
exifcommon "github.com/dsoprea/go-exif/v3/common"
jis "github.com/dsoprea/go-jpeg-image-structure/v2"
pis "github.com/dsoprea/go-png-image-structure/v2"
)
var ErrUnknownFileType = errors.New("can't scrub EXIF for this file type")
type ExifScrubber struct {
includedTagIds []uint16
includedTagPaths []string
}
func NewExifScrubber(includedTagIds []uint16, includedTagPaths []string) ExifScrubber {
return ExifScrubber{
includedTagIds: includedTagIds,
includedTagPaths: includedTagPaths,
}
}
func (scrubber *ExifScrubber) ScrubExif(fileData []byte) ([]byte, error) {
// Try scrubbing using JPEG package
jpegParser := jis.NewJpegMediaParser()
if jpegParser.LooksLikeFormat(fileData) {
intfc, err := jpegParser.ParseBytes(fileData)
if err != nil {
return nil, err
}
segmentList := intfc.(*jis.SegmentList)
rootIfd, _, err := segmentList.Exif()
if err != nil {
if err == exif.ErrNoExif {
// Incoming data contained no EXIF in the first place so we can return the original
return fileData, nil
}
return nil, err
}
filteredIb, err := scrubber.filteringIfdBuilder(rootIfd)
if err != nil {
return nil, err
}
segmentList.SetExif(filteredIb)
b := new(bytes.Buffer)
err = segmentList.Write(b)
if err != nil {
return nil, err
}
return b.Bytes(), nil
}
// Try scrubbing using PNG package
pngParser := pis.NewPngMediaParser()
if pngParser.LooksLikeFormat(fileData) {
intfc, err := pngParser.ParseBytes(fileData)
if err != nil {
return nil, err
}
chunks := intfc.(*pis.ChunkSlice)
rootIfd, _, err := chunks.Exif()
if err != nil {
if err == exif.ErrNoExif {
// Incoming data contained no EXIF in the first place so we can return the original
return fileData, nil
}
return nil, err
}
filteredIb, err := scrubber.filteringIfdBuilder(rootIfd)
if err != nil {
return nil, err
}
chunks.SetExif(filteredIb)
b := new(bytes.Buffer)
err = chunks.WriteTo(b)
if err != nil {
return nil, err
}
return b.Bytes(), nil
}
// Don't know how to handle other file formats, so we let the caller decide how to continue
return nil, ErrUnknownFileType
}
// Check whether the tag represented by `tag` is included in the path or tag ID list
func (scrubber *ExifScrubber) isTagAllowed(tag *exif.IfdTagEntry) bool {
// Check via IDs first (faster than string comparisons)
for _, includedId := range scrubber.includedTagIds {
if includedId == tag.TagId() {
return true
}
}
// If no IDs matched, also check IFD tag paths for inclusion
tagPath := fmt.Sprintf("%s/%s", tag.IfdPath(), tag.TagName())
for _, includedPath := range scrubber.includedTagPaths {
if includedPath == tagPath {
return true
}
}
return false
}
// This method follows the implementation of exif.NewIfdBuilderFromExistingChain()
func (scrubber *ExifScrubber) filteringIfdBuilder(rootIfd *exif.Ifd) (
firstIb *exif.IfdBuilder,
err error,
) {
var lastIb *exif.IfdBuilder
i := 0
for thisExistingIfd := rootIfd; thisExistingIfd != nil; thisExistingIfd = thisExistingIfd.NextIfd() {
// This only works when no non-standard mappings are used
ifdMapping, err := exifcommon.NewIfdMappingWithStandard()
if err != nil {
return nil, err
}
// This only works when no non-standard tags are used
tagIndex := exif.NewTagIndex()
err = exif.LoadStandardTags(tagIndex)
if err != nil {
return nil, err
}
newIb := exif.NewIfdBuilder(
ifdMapping,
tagIndex,
thisExistingIfd.IfdIdentity(),
thisExistingIfd.ByteOrder(),
)
if firstIb == nil {
firstIb = newIb
} else {
lastIb.SetNextIb(newIb)
}
err = scrubber.filteredAddTagsFromExisting(newIb, thisExistingIfd)
if err != nil {
return nil, err
}
lastIb = newIb
i++
}
return firstIb, nil
}
// This method follows the implementation of exif.IfdBuilder.AddTagsFromExisting()
func (scrubber *ExifScrubber) filteredAddTagsFromExisting(
ib *exif.IfdBuilder,
ifd *exif.Ifd,
) (err error) {
for i, ite := range ifd.Entries() {
if ite.IsThumbnailOffset() == true || ite.IsThumbnailSize() {
// These will be added on-the-fly when we encode.
continue
}
var bt *exif.BuilderTag
if ite.ChildIfdPath() != "" {
// If we want to add an IFD tag, we'll have to build it first and
// *then* add it via a different method.
// Figure out which of the child-IFDs that are associated with
// this IFD represents this specific child IFD.
var childIfd *exif.Ifd
for _, thisChildIfd := range ifd.Children() {
if thisChildIfd.ParentTagIndex() != i {
continue
} else if thisChildIfd.IfdIdentity().TagId() != 0xffff &&
thisChildIfd.IfdIdentity().TagId() != ite.TagId() {
fmt.Printf(
"child-IFD tag is not correct: TAG-POSITION=(%d) ITE=%s CHILD-IFD=%s\n",
thisChildIfd.ParentTagIndex(),
ite,
thisChildIfd,
)
}
childIfd = thisChildIfd
break
}
if childIfd == nil {
childTagIds := make([]string, len(ifd.Children()))
for j, childIfd := range ifd.Children() {
childTagIds[j] = fmt.Sprintf(
"0x%04x (parent tag-position %d)",
childIfd.IfdIdentity().TagId(),
childIfd.ParentTagIndex(),
)
}
fmt.Printf(
"could not find child IFD for child ITE: IFD-PATH=[%s] TAG-ID=(0x%04x) "+
"CURRENT-TAG-POSITION=(%d) CHILDREN=%v\n",
ite.IfdPath(),
ite.TagId(),
i,
childTagIds,
)
}
childIb, err := scrubber.filteringIfdBuilder(childIfd)
if err != nil {
return err
}
bt = ib.NewBuilderTagFromBuilder(childIb)
} else {
// Non-IFD tag.
isAllowed := scrubber.isTagAllowed(ite)
if !isAllowed {
continue
}
rawBytes, err := ite.GetRawBytes()
if err != nil {
return err
}
value := exif.NewIfdBuilderTagValueFromBytes(rawBytes)
bt = exif.NewBuilderTag(
ifd.IfdIdentity().UnindexedString(),
ite.TagId(),
ite.TagType(),
value,
ifd.ByteOrder(),
)
}
if bt.Value().IsBytes() {
err := ib.Add(bt)
if err != nil {
return err
}
} else if bt.Value().IsIb() {
err := ib.AddChildIb(bt.Value().Ib())
if err != nil {
return err
}
}
}
return nil
}

116
exifscrubber/exifscrubber_test.go Normal file
View file

@ -0,0 +1,116 @@
package exifscrubber
import (
"io/ioutil"
"log"
"testing"
"golang.org/x/exp/slices"
exif "github.com/dsoprea/go-exif/v3"
jis "github.com/dsoprea/go-jpeg-image-structure/v2"
pis "github.com/dsoprea/go-png-image-structure/v2"
)
func TestJpgFromFile(t *testing.T) {
// Read and strip original file
buf, err := ioutil.ReadFile("../fixtures/gps.jpg")
if err != nil {
t.Errorf("could not open file")
}
includeTagIds := []uint16{0x9209} // ID of "Flash" tag
includedPaths := []string{
"IFD/Orientation",
"IFD/GPSInfo/GPSTimeStamp",
"IFD/GPSInfo/GPSDateStamp",
}
scrubber := NewExifScrubber(includeTagIds[:], includedPaths[:])
updatedBuf, err := scrubber.ScrubExif(buf)
if err != nil {
log.Println(err)
}
// Check whether updated file only contains the specified paths
intfc, err := jis.NewJpegMediaParser().ParseBytes(updatedBuf)
if err != nil {
t.Errorf(err.Error())
}
sl := intfc.(*jis.SegmentList)
rootIfd, _, err := sl.Exif()
visitor := func(ifd *exif.Ifd, ite *exif.IfdTagEntry) error {
tagId := ite.TagId()
tagPath := ite.IfdPath() + "/" + ite.TagName()
tagContained := slices.Contains(includeTagIds, tagId)
pathContained := slices.Contains(includedPaths, tagPath)
contained := tagContained || pathContained
if !contained {
t.Errorf(
"tag %s (%d) included in EXIF although it hasn't been specified",
tagPath,
tagId,
)
}
return nil
}
rootIfd.EnumerateTagsRecursively(visitor)
}
func TestPngFromFile(t *testing.T) {
buf, err := ioutil.ReadFile("../fixtures/gps.png")
if err != nil {
t.Errorf("could not open file")
}
includeTagIds := []uint16{0x9209} // ID of "Flash" tag
includedPaths := []string{
"IFD/Orientation",
"IFD/GPSInfo/GPSTimeStamp",
"IFD/GPSInfo/GPSDateStamp",
}
scrubber := NewExifScrubber(includeTagIds[:], includedPaths[:])
updatedBuf, err := scrubber.ScrubExif(buf)
if err != nil {
log.Println(err)
}
// Check whether updated file only contains the specified paths
intfc, err := pis.NewPngMediaParser().ParseBytes(updatedBuf)
if err != nil {
t.Errorf(err.Error())
}
cs := intfc.(*pis.ChunkSlice)
rootIfd, _, err := cs.Exif()
visitor := func(ifd *exif.Ifd, ite *exif.IfdTagEntry) error {
tagId := ite.TagId()
tagPath := ite.IfdPath() + "/" + ite.TagName()
tagContained := slices.Contains(includeTagIds, tagId)
pathContained := slices.Contains(includedPaths, tagPath)
contained := tagContained || pathContained
if !contained {
t.Errorf(
"tag %s (%d) included in EXIF although it hasn't been specified",
tagPath,
tagId,
)
}
return nil
}
rootIfd.EnumerateTagsRecursively(visitor)
}

BIN
fixtures/gps.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 193 KiB

BIN
fixtures/gps.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 17 KiB

19
go.mod
View file

@ -1,3 +1,22 @@
module github.com/leon-richardt/jaf
go 1.19
require (
github.com/dsoprea/go-exif/v3 v3.0.0-20210512043655-120bcdb2a55e
github.com/dsoprea/go-jpeg-image-structure/v2 v2.0.0-20210512043942-b434301c6836
github.com/dsoprea/go-png-image-structure/v2 v2.0.0-20210512210324-29b889a6093d
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e
)
require (
github.com/dsoprea/go-iptc v0.0.0-20200609062250-162ae6b44feb // indirect
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd // indirect
github.com/dsoprea/go-photoshop-info-format v0.0.0-20200609050348-3db9b63b202c // indirect
github.com/dsoprea/go-utility/v2 v2.0.0-20200717064901-2fccff4aa15e // indirect
github.com/go-errors/errors v1.1.1 // indirect
github.com/go-xmlfmt/xmlfmt v0.0.0-20191208150333-d5b6f63a941b // indirect
github.com/golang/geo v0.0.0-20200319012246-673a6f80352d // indirect
golang.org/x/net v0.0.0-20200707034311-ab3426394381 // indirect
gopkg.in/yaml.v2 v2.3.0 // indirect
)

51
go.sum Normal file
View file

@ -0,0 +1,51 @@
github.com/dsoprea/go-exif/v2 v2.0.0-20200321225314-640175a69fe4/go.mod h1:Lm2lMM2zx8p4a34ZemkaUV95AnMl4ZvLbCUbwOvLC2E=
github.com/dsoprea/go-exif/v3 v3.0.0-20200717053412-08f1b6708903/go.mod h1:0nsO1ce0mh5czxGeLo4+OCZ/C6Eo6ZlMWsz7rH/Gxv8=
github.com/dsoprea/go-exif/v3 v3.0.0-20210428042052-dca55bf8ca15/go.mod h1:cg5SNYKHMmzxsr9X6ZeLh/nfBRHHp5PngtEPcujONtk=
github.com/dsoprea/go-exif/v3 v3.0.0-20210512043655-120bcdb2a55e h1:E4XTSQZF/JtOQWcSaJBJho7t+RNWfdO92W/5skg10Jk=
github.com/dsoprea/go-exif/v3 v3.0.0-20210512043655-120bcdb2a55e/go.mod h1:cg5SNYKHMmzxsr9X6ZeLh/nfBRHHp5PngtEPcujONtk=
github.com/dsoprea/go-iptc v0.0.0-20200609062250-162ae6b44feb h1:gwjJjUr6FY7zAWVEueFPrcRHhd9+IK81TcItbqw2du4=
github.com/dsoprea/go-iptc v0.0.0-20200609062250-162ae6b44feb/go.mod h1:kYIdx9N9NaOyD7U6D+YtExN7QhRm+5kq7//yOsRXQtM=
github.com/dsoprea/go-jpeg-image-structure/v2 v2.0.0-20210512043942-b434301c6836 h1:KGCiMMWxODEMmI3+9Ms04l73efoqFVNKKKPbVyOvKrU=
github.com/dsoprea/go-jpeg-image-structure/v2 v2.0.0-20210512043942-b434301c6836/go.mod h1:WaARaUjQuSuDCDFAiU/GwzfxMTJBulfEhqEA2Tx6B4Y=
github.com/dsoprea/go-logging v0.0.0-20190624164917-c4f10aab7696/go.mod h1:Nm/x2ZUNRW6Fe5C3LxdY1PyZY5wmDv/s5dkPJ/VB3iA=
github.com/dsoprea/go-logging v0.0.0-20200517223158-a10564966e9d/go.mod h1:7I+3Pe2o/YSU88W0hWlm9S22W7XI1JFNJ86U0zPKMf8=
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd h1:l+vLbuxptsC6VQyQsfD7NnEC8BZuFpz45PgY+pH8YTg=
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd/go.mod h1:7I+3Pe2o/YSU88W0hWlm9S22W7XI1JFNJ86U0zPKMf8=
github.com/dsoprea/go-photoshop-info-format v0.0.0-20200609050348-3db9b63b202c h1:7j5aWACOzROpr+dvMtu8GnI97g9ShLWD72XIELMgn+c=
github.com/dsoprea/go-photoshop-info-format v0.0.0-20200609050348-3db9b63b202c/go.mod h1:pqKB+ijp27cEcrHxhXVgUUMlSDRuGJJp1E+20Lj5H0E=
github.com/dsoprea/go-png-image-structure/v2 v2.0.0-20210512210324-29b889a6093d h1:2zNIgrJTspLxUKoJGl0Ln24+hufPKSjP3cu4++5MeSE=
github.com/dsoprea/go-png-image-structure/v2 v2.0.0-20210512210324-29b889a6093d/go.mod h1:scnx0wQSM7UiCMK66dSdiPZvL2hl6iF5DvpZ7uT59MY=
github.com/dsoprea/go-utility v0.0.0-20200711062821-fab8125e9bdf/go.mod h1:95+K3z2L0mqsVYd6yveIv1lmtT3tcQQ3dVakPySffW8=
github.com/dsoprea/go-utility/v2 v2.0.0-20200717064901-2fccff4aa15e h1:IxIbA7VbCNrwumIYjDoMOdf4KOSkMC6NJE4s8oRbE7E=
github.com/dsoprea/go-utility/v2 v2.0.0-20200717064901-2fccff4aa15e/go.mod h1:uAzdkPTub5Y9yQwXe8W4m2XuP0tK4a9Q/dantD0+uaU=
github.com/go-errors/errors v1.0.1/go.mod h1:f4zRHt4oKfwPJE5k8C9vpYG+aDHdBFUsgrm6/TyX73Q=
github.com/go-errors/errors v1.0.2/go.mod h1:psDX2osz5VnTOnFWbDeWwS7yejl+uV3FEWEp4lssFEs=
github.com/go-errors/errors v1.1.1 h1:ljK/pL5ltg3qoN+OtN6yCv9HWSfMwxSx90GJCZQxYNg=
github.com/go-errors/errors v1.1.1/go.mod h1:psDX2osz5VnTOnFWbDeWwS7yejl+uV3FEWEp4lssFEs=
github.com/go-xmlfmt/xmlfmt v0.0.0-20191208150333-d5b6f63a941b h1:khEcpUM4yFcxg4/FHQWkvVRmgijNXRfzkIDHh23ggEo=
github.com/go-xmlfmt/xmlfmt v0.0.0-20191208150333-d5b6f63a941b/go.mod h1:aUCEOzzezBEjDBbFBoSiya/gduyIiWYRP6CnSFIV8AM=
github.com/golang/geo v0.0.0-20190916061304-5b978397cfec/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI=
github.com/golang/geo v0.0.0-20200319012246-673a6f80352d h1:C/hKUcHT483btRbeGkrRjJz+Zbcj8audldIi9tRJDCc=
github.com/golang/geo v0.0.0-20200319012246-673a6f80352d/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e/go.mod h1:Kr81I6Kryrl9sr8s2FK3vxD90NdsKWRuOIl2O4CvYbA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200320220750-118fecf932d8/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200707034311-ab3426394381 h1:VXak5I6aEWmAXeQjA+QSZzlgNrpq9mjcfDemuexIKsU=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=

17
jaf.go
View file

@ -7,13 +7,13 @@ import (
"math/rand"
"net/http"
"time"
"github.com/leon-richardt/jaf/exifscrubber"
)
const allowedChars = "0123456789ABCDEFGHIJKLMNOPQRTSUVWXYZabcdefghijklmnopqrstuvwxyz"
var (
config Config
)
var config Config
type parameters struct {
configFile string
@ -40,6 +40,15 @@ func main() {
log.Fatalf("could not read config file: %s\n", err.Error())
}
handler := uploadHandler{
config: config,
}
if config.ScrubExif {
scrubber := exifscrubber.NewExifScrubber(config.ExifAllowedIds, config.ExifAllowedPaths)
handler.exifScrubber = &scrubber
}
// Start server
uploadServer := &http.Server{
ReadTimeout: 30 * time.Second,
@ -48,6 +57,6 @@ func main() {
}
log.Printf("starting jaf on port %d\n", config.Port)
http.Handle("/upload", &uploadHandler{config: config})
http.Handle("/upload", &handler)
uploadServer.ListenAndServe()
}

74
uploadhandler.go
View file

@ -5,17 +5,19 @@ import (
"io"
"log"
"math/rand"
"mime/multipart"
"net/http"
"os"
"strings"
"github.com/leon-richardt/jaf/exifscrubber"
)
type uploadHandler struct {
config *Config
config *Config
exifScrubber *exifscrubber.ExifScrubber
}
func (h *uploadHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
func (handler *uploadHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
uploadFile, header, err := r.FormFile("file")
@ -24,10 +26,47 @@ func (h *uploadHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
log.Println(" could not read uploaded file: " + err.Error())
return
}
defer uploadFile.Close()
fileData, err := io.ReadAll(uploadFile)
uploadFile.Close()
if err != nil {
http.Error(w, "could not read attached file: "+err.Error(), http.StatusInternalServerError)
log.Println(" could not read attached file: " + err.Error())
return
}
// Scrub EXIF, if requested and detectable by us
if handler.config.ScrubExif {
scrubbedData, err := handler.exifScrubber.ScrubExif(fileData[:])
if err == nil {
// If scrubbing was successful, update what to write to file
fileData = scrubbedData
} else {
// Unknown file types (not PNG or JPEG) are allowed to contain EXIF, as we don't know
// how to handle them. Handling of other errors depends on configuration.
if err != exifscrubber.ErrUnknownFileType {
if handler.config.ExifAbortOnError {
log.Printf("could not scrub EXIF from file, aborting upload: %s", err.Error())
http.Error(
w,
"could not scrub EXIF from file: "+err.Error(),
http.StatusInternalServerError,
)
return
}
// An error occured but we are configured to proceed with the upload anyway
log.Printf(
"could not scrub EXIF from file but proceeding with upload as configured: %s",
err.Error(),
)
}
}
}
_, fileExtension := splitFileName(header.Filename)
link, err := generateLink(h, &uploadFile, fileExtension)
link, err := generateLink(handler, fileData[:], fileExtension)
if err != nil {
http.Error(w, "could not save file: "+err.Error(), http.StatusInternalServerError)
log.Println(" could not save file: " + err.Error())
@ -41,7 +80,7 @@ func (h *uploadHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Generates a valid link to uploadFile with the specified file extension.
// Returns the link or an error in case of failure.
// Does not close the passed file pointer.
func generateLink(handler *uploadHandler, uploadFile *multipart.File, fileExtension string) (string, error) {
func generateLink(handler *uploadHandler, fileData []byte, fileExtension string) (string, error) {
// Find an unused file name
var fullFileName string
var savePath string
@ -57,7 +96,7 @@ func generateLink(handler *uploadHandler, uploadFile *multipart.File, fileExtens
link := handler.config.LinkPrefix + fullFileName
err := saveFile(uploadFile, savePath)
err := saveFile(fileData[:], savePath)
if err != nil {
return "", err
}
@ -65,24 +104,13 @@ func generateLink(handler *uploadHandler, uploadFile *multipart.File, fileExtens
return link, nil
}
func saveFile(data *multipart.File, name string) error {
file, err := os.Create(name)
if err != nil {
return err
}
defer file.Close()
_, err = io.Copy(file, *data)
if err != nil {
return err
}
return nil
func saveFile(fileData []byte, name string) error {
err := os.WriteFile(name, fileData, 0o644)
return err
}
func fileExists(filename string) bool {
_, err := os.Stat(filename)
func fileExists(fileName string) bool {
_, err := os.Stat(fileName)
return !errors.Is(err, os.ErrNotExist)
}