Update README.md

add documentation for wrappers
This commit is contained in:
skeris 2022-07-10 18:49:54 +03:00 committed by GitHub
parent 976324112b
commit c4861d422f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -3,7 +3,6 @@
subj is a log collecting service for large microservice systems. It provides an grpc inteface for communicating with log emitters. I've choose grpc because it pretty nice, does't need text data overhead due to data transporting and most of backend languages has own protoc to generate clients using provided .proto file. For statistics database TL use clickhouse as one of the best OLAP DB. On the other hand was Apache Cassandra but ClickHouse a way more flexible. If ClickHouse is down TL store unprocessed records to local bbolt storage. Clickhouse has some constrains on inserting such as it may crush on multiple discrete inserts, so I implement batch inserting every second
## Potential troubles
- sometimes clickhouse broke connection without known reason. I'm try to fix this but this trouble is still exists
- with large amount of incoming data batch may retard. I mean that there are may be more than one second to batch iteration. I have implementation of zero allocation workerpool and I can use it there but it is static workerpool without growing and it may create the same bottleneck but wider(n times wider, where n is workerpool size). On the other hand I can create goroutine for each batch to avoid retarding, but it consume more RAM. The worst of is is that the more retarding batch is than larger next batch
## Requirements
@ -39,3 +38,65 @@ there are list of acceptable types for fields
### Storing logrecords
I think that it is bad idea to send an unary request for every logrecord to store when we can use **streams** isn't it? so just create one **Valve** stream and send data in it every time when you need to store log event
# WRAPPERS
TL provide some zap.LoggerCore wrappers to make using of trashlog more convinient and simple. There are two ways of logging - to telegram channel by BotAPI or to trashlog server. Strongly recommended use github.com/themakers/hlog as log formatter
## ZAPTG
to use it just create a new zaptg core and wrap old logger with new core.
ctx - current app context
zap.InfoLevel - min loglevel to store
<bot token> - token priveded by BotFather
version.Release - version of app. should use git tag
version.Commit - current commit
0 - build time
```
tel, err := zaptg.NewCore(
ctx,
zap.InfoLevel,
"<bot token>",
version.Release,
version.Commit,
0,
<channel id>,
)
if err != nil {
panic(err)
}
logger = logger.WithOptions(zap.WrapCore(func(core zapcore.Core) zapcore.Core {
return zapcore.NewTee(core, tel)
}))
```
## ZAPTRASHLOG
Wrapper to send data to trashlog server
to use it just create a new zaptg core and wrap old logger with new core.
ctx - current app context
zap.InfoLevel - min loglevel to store
<bot token> - token priveded by BotFather
v.Release - version of app. should use git tag
v.Commit - current commit
time.Now().Unix() - run time
```
tl, err := zaptrashlog.NewCore(
ctx,
zapcore.DebugLevel,
"<trashlog uri>",
v.Release,
v.Commit,
time.Now().Unix(),
)
if err != nil {
panic(err)
}
logger = logger.WithOptions(zap.WrapCore(func(core zapcore.Core) zapcore.Core {
return zapcore.NewTee(core, tel)
}))
```