k-space
/
microscript-base
8
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Compare commits

merge into: k-space:master
Branches Tags
k-space:leonid98
k-space:master
...
pull from: k-space:leonid98
Branches Tags
k-space:leonid98
k-space:master

No commits in common. 'master' and 'leonid98' have entirely different histories.
master ... leonid98

10 changed files with 393 additions and 24 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats
  1. 18
      .drone.yml
  2. 6
      .flake8
  3. 9
      .gitlint
  4. 16
      .pre-commit-config.yaml
  5. 17
      Dockerfile
  6. 21
      README.md
  7. 27
      docker-compose.yml
  8. 61
      main.py
  9. 235
      mock_response.py
  10. 7
      ssh_config

18
.drone.yml
Unescape View File

@ -1,2 +1,16 @@
kind: template
load: docker.yaml
---
kind: pipeline
type: kubernetes
name: default
steps:
- name: docker
image: plugins/docker
settings:
repo: harbor.k-space.ee/${DRONE_REPO}
registry: harbor.k-space.ee
mtu: 1300
username:
from_secret: docker_username
password:
from_secret: docker_password

6
.flake8
Unescape View File

@ -0,0 +1,6 @@
[flake8]
inline-quotes = "
multiline-quotes = """
indent-size = 4
max-line-length = 160
ignore = Q003 E128 E704 E731

9
.gitlint
Unescape View File

@ -0,0 +1,9 @@
[general]
ignore=body-is-missing,T3
ignore-stdin=true
[title-match-regex]
regex=[A-Z]
[author-valid-email]
regex=[^@]+@k-space.ee

16
.pre-commit-config.yaml
Unescape View File

@ -0,0 +1,16 @@
repos:
- repo: https://github.com/PyCQA/flake8
rev: 3.9.2
hooks:
- id: flake8
additional_dependencies: [flake8-typing-imports==1.10.0,flake8-quotes==3.2.0]
- repo: https://github.com/jorisroovers/gitlint
rev: v0.15.1
hooks:
- id: gitlint
- repo: https://github.com/Lucas-C/pre-commit-hooks-nodejs
rev: v1.1.1
hooks:
- id: dockerfile_lint

17
Dockerfile
Unescape View File

@ -1,16 +1,7 @@
FROM python:alpine
RUN apk add git bash openssh curl
RUN pip install pymongo
ADD ssh_config /etc/ssh/
# Inject public SSH key of git.k-space.ee
RUN bash -c "ssh-keyscan -t ecdsa git.k-space.ee >> /etc/ssh/ssh_known_hosts"
# For git commits use user `minion`, note that this is distinct from user
# used for SSH authentication, including one going towards git.k-space.ee
RUN git config --global user.email "minion@k-space.ee"
RUN git config --global user.name "Minion"
LABEL name="k-space/microscript-base" \
LABEL name="k-space/microscript-skeleton" \
version="rc" \
maintainer="Lauri Võsandi <lauri@k-space.ee>"
RUN pip install pymongo
ADD main.py /main.py
CMD /main.py

21
README.md
Unescape View File

@ -8,5 +8,22 @@ In this particular example MongoDB is included in the setup.
# How to use this repository
Just mount scripts using ConfigMap and run
`harbor.k-space.ee/k-space/microscript-base` directly from Kubernetes
* Fork with suitable name under `k-space` organization
* Adjust maintainer name and e-mail in `Dockerfile
* Add/remove dependencies as necessary in `Dockerfile`
* Make sure `docker-compose.yml` brings up sensible development environment
# Prerequisites
* Docker runtime is installed
* Your user is user of group `docker`
* Docker compose is installed
* Pre-commit framework is installed, if necessary `apt-get install pre-commit`
# Development environment
```
docker-compose up --build
```
To access MongoExpress open up http://localhost:8081/

27
docker-compose.yml
Unescape View File

@ -0,0 +1,27 @@
version: '3.7'
services:
main:
build:
context: .
mongo:
logging:
driver: none
image: mongo
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
mongo-express:
logging:
driver: none
image: mongo-express
restart: always
ports:
- 8081:8081
environment:
ME_CONFIG_MONGODB_ADMINUSERNAME: root
ME_CONFIG_MONGODB_ADMINPASSWORD: example
ME_CONFIG_MONGODB_URL: mongodb://root:example@mongo:27017/

61
main.py
Unescape View File

@ -0,0 +1,61 @@
from flask import Flask
from mock_response import MOCK_RESPONSE
app = Flask(__name__)
@app.route("/")
def index():
return MOCK_RESPONSE
if __name__ == "__main__":
app.run(port=8080)
def transactions_pull(mongodb):
if not const.LHV_IMPORT:
return ""
last_timestamps = {}
for self_account in SELF_ACCOUNTS:
last_timestamps[self_account] = date(2015, 1, 1)
for row in mongodb.cashflow.find({"type": "transaction", "self_account": self_account}).sort([("_id",-1)]).limit(1):
last_timestamps[self_account] = datetime.strptime(row["timestamp"], "%Y-%m-%d")
today = date.today()
for self_account in SELF_ACCOUNTS:
created = datetime.now().strftime("%Y-%m-%dT%H:%M:%S.000")
start = last_timestamps[self_account].strftime("%Y-%m-%d")
finish = today.strftime("%Y-%m-%d")
# zomg what
while True:
response = requests.get("https://connect.lhv.eu/messages/next", cert=KEYPAIR)
msg_id = response.headers.get("Message-Response-Id")
if not msg_id:
break
print("Discarding:", msg_id)
response = requests.delete("https://connect.lhv.eu/messages/%s" % msg_id, cert=KEYPAIR)
response = requests.post("https://connect.lhv.eu/account-statement", data=BUF % locals(), cert=KEYPAIR)
sleep(2)
response = requests.get("https://connect.lhv.eu/messages/next", cert=KEYPAIR)
buf = response.content
if not buf:
raise ValueError("Failed to retrieve next %s" % response.status_code)
for values in import_statements(buf):
mongodb.cashflow.update_one({
"type": "transaction",
"reference": values[0]
}, {
"$set": {
"timestamp": values[1],
"self_account": values[2],
"peer_account": values[3],
"peer_name": values[4],
"peer_name_normalized": norm(normalize_legal_entity(values[4])),
"credit_debit": values[5],
"amount": values[6],
"comment": values[7]
},
}, upsert=True)
auto_associate(mongodb)
requests.get("https://hc-ping.com/375391a5-ff13-47e3-8892-876d1d995058")
return "ok"

235
mock_response.py
Unescape View File

@ -0,0 +1,235 @@
MOCK_RESPONSE = """<?xml version="1.0" encoding="UTF-8"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02 camt.053.001.02.xsd">
<BkToCstmrStmt>
<GrpHdr>
<MsgId>180114572</MsgId>
<CreDtTm>2022-04-22T00:29:16.262806</CreDtTm>
</GrpHdr>
<Stmt>
<Id>180114572EUR</Id>
<CreDtTm>2022-04-22T00:29:16.262806</CreDtTm>
<FrToDt>
<FrDtTm>2022-04-21T01:00:00</FrDtTm>
<ToDtTm>2022-04-22T00:29:15</ToDtTm>
</FrToDt>
<Acct>
<Id>
<IBAN>EE467700771002926893</IBAN>
</Id>
<Ccy>EUR</Ccy>
<Svcr>
<FinInstnId>
<BIC>LHVBEE22</BIC>
<Nm>AS LHV Pank</Nm>
<PstlAdr>
<AdrTp>BIZZ</AdrTp>
<StrtNm>Tartu mnt.</StrtNm>
<BldgNb>2</BldgNb>
<PstCd>10145</PstCd>
<TwnNm>Tallinn</TwnNm>
<CtrySubDvsn>Harjumaa</CtrySubDvsn>
<Ctry>EE</Ctry>
</PstlAdr>
</FinInstnId>
</Svcr>
</Acct>
<Bal>
<Tp>
<CdOrPrtry>
<Cd>OPBD</Cd>
</CdOrPrtry>
</Tp>
<Amt Ccy="EUR">0.00</Amt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Dt>
<Dt>2022-04-21</Dt>
</Dt>
</Bal>
<Bal>
<Tp>
<CdOrPrtry>
<Cd>CLBD</Cd>
</CdOrPrtry>
</Tp>
<Amt Ccy="EUR">64.00</Amt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Dt>
<Dt>2022-04-22</Dt>
</Dt>
</Bal>
<TxsSummry>
<TtlCdtNtries>
<NbOfNtries>2</NbOfNtries>
<Sum>64.00</Sum>
</TtlCdtNtries>
<TtlDbtNtries>
<NbOfNtries>0</NbOfNtries>
<Sum>0.00</Sum>
</TtlDbtNtries>
</TxsSummry>
<Ntry>
<Amt Ccy="EUR">35.00</Amt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Sts>BOOK</Sts>
<BookgDt>
<Dt>2022-04-21</Dt>
</BookgDt>
<AcctSvcrRef>C3D9FD211FC1EC11A2E400155DA95D0B</AcctSvcrRef>
<BkTxCd>
<Domn>
<Cd>PMNT</Cd>
<Fmly>
<Cd>RCDT</Cd>
<SubFmlyCd>OTHR</SubFmlyCd>
</Fmly>
</Domn>
<Prtry>
<Cd>INTERNAL</Cd>
</Prtry>
</BkTxCd>
<NtryDtls>
<TxDtls>
<Refs>
<AcctSvcrRef>C3D9FD211FC1EC11A2E400155DA95D0B</AcctSvcrRef>
<InstrId>31</InstrId>
</Refs>
<AmtDtls>
<InstdAmt>
<Amt Ccy="EUR">35.00</Amt>
</InstdAmt>
<TxAmt>
<Amt Ccy="EUR">35.00</Amt>
</TxAmt>
</AmtDtls>
<RltdPties>
<Dbtr>
<Nm>Rasmus Kallas</Nm>
<Id>
<PrvtId>
<Othr>
<Id>50304110853</Id>
<SchmeNm>
<Cd>NIDN</Cd>
</SchmeNm>
</Othr>
</PrvtId>
</Id>
</Dbtr>
<DbtrAcct>
<Id>
<IBAN>EE677700771002886454</IBAN>
</Id>
</DbtrAcct>
<Cdtr>
<Nm>K-SPACE MTÜ</Nm>
</Cdtr>
<CdtrAcct>
<Id>
<IBAN>EE467700771002926893</IBAN>
</Id>
</CdtrAcct>
</RltdPties>
<RltdAgts>
<DbtrAgt>
<FinInstnId>
<BIC>LHVBEE22</BIC>
<Nm>AS LHV Pank</Nm>
</FinInstnId>
</DbtrAgt>
<CdtrAgt>
<FinInstnId>
<BIC>LHVBEE22</BIC>
<Nm>AS LHV Pank</Nm>
</FinInstnId>
</CdtrAgt>
</RltdAgts>
<RmtInf>
<Ustrd>Membership</Ustrd>
</RmtInf>
</TxDtls>
</NtryDtls>
</Ntry>
<Ntry>
<Amt Ccy="EUR">29.00</Amt>
<CdtDbtInd>CRDT</CdtDbtInd>
<Sts>BOOK</Sts>
<BookgDt>
<Dt>2022-04-21</Dt>
</BookgDt>
<AcctSvcrRef>1A79D7E887C1EC11A2E400155DA95D0B</AcctSvcrRef>
<BkTxCd>
<Domn>
<Cd>PMNT</Cd>
<Fmly>
<Cd>RCDT</Cd>
<SubFmlyCd>OTHR</SubFmlyCd>
</Fmly>
</Domn>
<Prtry>
<Cd>INST</Cd>
</Prtry>
</BkTxCd>
<NtryDtls>
<TxDtls>
<Refs>
<AcctSvcrRef>1A79D7E887C1EC11A2E400155DA95D0B</AcctSvcrRef>
<PmtInfId>RD230928989</PmtInfId>
<InstrId>294</InstrId>
</Refs>
<AmtDtls>
<InstdAmt>
<Amt Ccy="EUR">29.00</Amt>
</InstdAmt>
<TxAmt>
<Amt Ccy="EUR">29.00</Amt>
</TxAmt>
</AmtDtls>
<RltdPties>
<Dbtr>
<Nm>JOON OTT</Nm>
<Id>
<PrvtId>
<Othr>
<Id>39907180819</Id>
<SchmeNm>
<Cd>NIDN</Cd>
</SchmeNm>
</Othr>
</PrvtId>
</Id>
</Dbtr>
<DbtrAcct>
<Id>
<IBAN>EE531010010557795013</IBAN>
</Id>
</DbtrAcct>
<Cdtr>
<Nm>K-SPACE MTÜ</Nm>
</Cdtr>
<CdtrAcct>
<Id>
<IBAN>EE467700771002926893</IBAN>
</Id>
</CdtrAcct>
</RltdPties>
<RltdAgts>
<DbtrAgt>
<FinInstnId>
<BIC>EEUHEE2X</BIC>
</FinInstnId>
</DbtrAgt>
<CdtrAgt>
<FinInstnId>
<BIC>LHVBEE22</BIC>
</FinInstnId>
</CdtrAgt>
</RltdAgts>
<RmtInf>
<Ustrd>Regular tier</Ustrd>
</RmtInf>
</TxDtls>
</NtryDtls>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>"""

7
ssh_config
Unescape View File

@ -1,7 +0,0 @@
Host *
IdentitiesOnly yes
IdentityFile /config/ssh_identity
UserKnownHostsFile /config/ssh_known_hosts
# TODO: Remove following after ROS7 upgrades
HostKeyAlgorithms +ssh-rsa
PubkeyAcceptedKeyTypes +ssh-rsa
Write Preview
Loading…
Cancel
Save