Migration redesign (#119)

* migration redesign init

* new color picker

* changelog

* added UI language selection

* fix layout issue on recipe editor

* remove git as dependency

* added UI editor for original URL

* CI/CD Tests

* test: fixed migration routes

Co-authored-by: Hayden <hay-kot@pm.me>
This commit is contained in:
Hayden 2021-01-23 19:53:39 -09:00 committed by GitHub
commit 079ebd8ee1
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
32 changed files with 549 additions and 367 deletions

View file

@ -1,49 +0,0 @@
name: Docker Build Alpine
on:
push:
branches:
- new-tests
jobs:
build:
runs-on: ubuntu-latest
steps:
#
# Checkout
#
- name: checkout code
uses: actions/checkout@v2
#
# Setup QEMU
#
- name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v1
# with:
# image: tonistiigi/binfmt:latest
# platforms: all
#
# Setup Buildx
#
- name: install buildx
id: buildx
uses: docker/setup-buildx-action@v1
with:
install: true
#
# Login to Docker Hub
#
- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
#
# Build
#
- name: build the image
run: |
docker build --push \
--tag hkotel/mealie:alpine \
--platform linux/amd64,linux/arm/v7,linux/arm64 .

View file

@ -30,15 +30,15 @@ jobs:
with:
virtualenvs-create: true
virtualenvs-in-project: true
#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
# #----------------------------------------------
# # load cached venv if cache exists
# #----------------------------------------------
# - name: Load cached venv
# id: cached-poetry-dependencies
# uses: actions/cache@v2
# with:
# path: .venv
# key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------

View file

@ -16,15 +16,7 @@ Don't forget to [join the Discord](https://discord.gg/R6QDyJgbD2)!
# Todo's
Documentation
- [ ] V0.1.0 Release Notes
- [ ] Nextcloud Migration How To
- [ ] New Docker Setup with Sqlite
- [ ] Update Env Variables
- [ ] New Roadmap / Milestones
Frontend
- [x] Prep / Cook / Total Time Indicator + Editor
- [ ] No Meal Today Page instead of Null
- [ ] Recipe Print Page
- [ ] Recipe Editor Data Validation Client Side
@ -32,12 +24,7 @@ Frontend
- [ ] Advanced Search Page, draft started
- [ ] Filter by Category
- [ ] Filter by Tags
- [ ] Search Bar redesign
- [x] Initial
- [ ] Results redesign
- [x] Replace Backups card with something like Home Assistant
- [x] Replace import card with something like Home Assistant
- [x] Select which imports to do
- [ ] Search Bar Results Redesign
Backend
- [ ] Database Import
@ -46,11 +33,10 @@ Backend
- [ ] Meal Plans
- [x] Settings
- [x] Themes
- [x] Remove Print / Debug Code
- [ ] Remove Print / Debug Code
- [ ] Support how to sections and how to steps
- [ ] Recipe request by category/tags
SQL
- [ ] Setup Database Migrations

BIN
dev/favicon.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 7 KiB

View file

@ -7,16 +7,22 @@
- Fixed failed database initialization at startup
- Fixed misaligned text on various cards
- Fixed bug that blocked opening links in new tabs
- Fixed router link bugs - Issue #122
### Features and Improvements
- UI Language Selection
- Meal Planner
- Improved Search (Fuzzy Search)
- New Scheduled card support
- Upload/Download backups
- Dockerfile now 1/5 of the size!
- **Minor**
- Migrations
- Card based redesign
- Upload from the UI
- Unified Chowdown/Nextcloud import process.
- Continued work on button/style unification
- Adding icons to buttons
- New Color Theme Picker UI
### Development
- Fixed Vetur config file. Autocomplete in VSCode works!

View file

@ -1,7 +1,7 @@
# Meal Planner
## Working with Meal Plans
In Mealie you can create a mealplan based off the calendar inputs on the meal planner page. There is no limit to how long or how short a meal plan is. You may also create duplicate meal plans for the same date range. After selecting your date range, click on the card for each day and seach through recipes to find your choice. After selecting a recipe for all meals save the plan. You can also randomly generate meal plans.
In Mealie you can create a mealplan based off the calendar inputs on the meal planner page. There is no limit to how long or how short a meal plan is. You may also create duplicate meal plans for the same date range. After selecting your date range, click on the card for each day and search through recipes to find your choice. After selecting a recipe for all meals save the plan. You can also randomly generate meal plans.
To edit the meal in a meal plan simply select the edit button on the card in the timeline. Similiarly, to delete a mealplan click the delete button on the card in the timeline. Currently there is no support to change the date range in a meal plan.

View file

@ -1,13 +1,13 @@
# Migration
### Chowdown
## Chowdown
To migrate recipes from a Chowdown
1. Download the code repository as a .zip file
2. Upload the .zip file in the Chowdown section in Mealie
3. Select import on the newly available migration.
In the Admin page on the in the Migration section you can provide a URL for a repo hosting a [Chowdown](https://github.com/clarklab/chowdown) repository and Mealie will pull the images and recipes from the instance and automatically import them into the database. Due to the nature of the yaml format you may have mixed results but you should get an error report of the recipes that had errors and will need to be manually added. Note that you can only import the repo as a whole. You cannot import individual recipes.
We'd like to support additional migration paths. [See open issues.](https://github.com/hay-kot/mealie/issues)
### Nextcloud Recipes
Nextcloud recipes can be imported from either a zip file the contains the data stored in Nextcloud. The zip file can be uploaded from the frontend or placed in the data/migrations/Nextcloud directory. See the example folder structure below to ensure your recipes are able to be imported.
## Nextcloud Recipes
Nextcloud recipes can be imported from a zip file the contains the data stored in Nextcloud. The zip file can be uploaded from the frontend or placed in the data/migrations/nextcloud directory. See the example folder structure below to ensure your recipes are able to be imported.
```
nextcloud_recipes.zip
@ -21,6 +21,3 @@ nextcloud_recipes.zip
└── recipe_3
└── recipe.json
```
**Currently Proposed Are:**
- Open Eats

View file

@ -38,7 +38,7 @@ Feature placement is not set in stone. This is much more of a guideline than any
- [ ] Additional Backup / Import Features
- [ ] Import Recipes Force/Rebase options
- [x] Upload .zip file
- [ ] Improved Color Picker
- [x] Improved Color Picker
- [x] Meal Plan redesign
### Backend
- [ ] PostgreSQL Support

View file

@ -1,3 +0,0 @@
{
"include": ["./src/**/*"]
}

View file

@ -58,6 +58,7 @@ export default {
mounted() {
this.$store.dispatch("initTheme")
this.$store.dispatch("requestRecentRecipes")
this.$store.dispatch("initLang")
this.darkModeSystemCheck()
this.darkModeAddEventListener()
},

View file

@ -2,42 +2,27 @@ import { baseURL } from "./api-utils";
import { apiReq } from "./api-utils";
import { store } from "../store/store";
const migrationBase = baseURL + "migration/";
const migrationBase = baseURL + "migrations/";
const migrationURLs = {
upload: migrationBase + "upload/",
delete: (file) => `${migrationBase}${file}/delete/`,
chowdownURL: migrationBase + "chowdown/repo/",
nextcloudAvaiable: migrationBase + "nextcloud/available/",
nextcloudImport: (selection) =>
`${migrationBase}nextcloud/${selection}/import/`,
// New
all: migrationBase,
delete: (folder, file) => `${migrationBase}/${folder}/${file}/delete/`,
import: (folder, file) => `${migrationBase}/${folder}/${file}/import/`,
};
export default {
async migrateChowdown(repoURL) {
let postBody = { url: repoURL };
let response = await apiReq.post(migrationURLs.chowdownURL, postBody);
async getMigrations() {
let response = await apiReq.get(migrationURLs.all);
return response.data;
},
async delete(folder, file) {
let response = await apiReq.delete(migrationURLs.delete(folder, file));
return response.data;
},
async import(folder, file) {
let response = await apiReq.post(migrationURLs.import(folder, file));
store.dispatch("requestRecentRecipes");
return response.data;
},
async getNextcloudImports() {
let response = await apiReq.get(migrationURLs.nextcloudAvaiable);
return response.data;
},
async importNextcloud(selected) {
let response = await apiReq.post(migrationURLs.nextcloudImport(selected));
return response.data;
},
async uploadFile(form_data) {
let response = await apiReq.post(migrationURLs.upload, form_data, {
headers: {
"Content-Type": "multipart/form-data",
},
});
return response.data;
},
async delete(file_folder_name) {
let response = await apiReq.delete(migrationURLs.delete(file_folder_name));
return response.data;
},
};

View file

@ -12,6 +12,7 @@
></v-file-input>
</v-col>
<v-col cols="3"></v-col>
</v-row>
<v-row>
<v-col>
<v-text-field
@ -32,7 +33,6 @@
></v-text-field
></v-col>
</v-row>
</v-row>
<v-text-field
class="my-3"
:label="$t('recipe.recipe-name')"
@ -206,6 +206,11 @@
<v-icon>mdi-plus</v-icon>
</v-btn>
<BulkAdd @bulk-data="appendSteps" />
<v-text-field
v-model="value.orgURL"
class="mt-10"
label="Original URL"
></v-text-field>
</v-col>
</v-row>
</v-card-text>

View file

@ -16,11 +16,11 @@
v-for="backup in backups"
:key="backup.name"
>
<v-card @click="openDialog(backup)">
<v-card hover outlined @click="openDialog(backup)">
<v-card-text>
<v-row align="center">
<v-col cols="12" sm="2">
<v-icon color="primary"> mdi-backup-restore </v-icon>
<v-icon large color="primary"> mdi-backup-restore </v-icon>
</v-col>
<v-col cols="12" sm="10">
<div>

View file

@ -34,6 +34,8 @@
:backups="availableBackups"
/>
<SuccessFailureAlert
ref="report"
title="Back Restore Report"
success-header="Successfully Imported"
:success="successfulImports"
failed-header="Failed Imports"
@ -91,6 +93,7 @@ export default {
this.backupLoading = false;
this.successfulImports = successful;
this.failedImports = failed;
this.$refs.report.open();
},
},
};

View file

@ -0,0 +1,50 @@
<template>
<v-card>
<v-card-title> General Settings </v-card-title>
<v-divider></v-divider>
<v-card-text>
<v-row>
<v-col>
<v-select
v-model="selectedLang"
:items="langOptions"
item-text="name"
item-value="value"
label="Language"
>
</v-select>
</v-col>
<v-spacer></v-spacer>
<v-spacer></v-spacer>
</v-row>
</v-card-text>
</v-card>
</template>
<script>
export default {
data() {
return {
langOptions: [],
selectedLang: "en",
};
},
mounted() {
this.getOptions();
},
watch: {
selectedLang() {
this.$store.commit("setLang", this.selectedLang);
},
},
methods: {
getOptions() {
this.langOptions = this.$store.getters.getAllLangs;
this.selectedLang = this.$store.getters.getActiveLang;
},
},
};
</script>
<style>
</style>

View file

@ -0,0 +1,97 @@
<template>
<v-card class="my-2" :loading="loading">
<v-card-title>
{{ title }}
<v-spacer></v-spacer>
<span>
<UploadBtn
class="mt-1"
:url="`/api/migrations/${folder}/upload/`"
@uploaded="$emit('refresh')"
/>
</span>
</v-card-title>
<v-card-text> {{ description }}</v-card-text>
<div v-if="available[0]">
<v-card
outlined
v-for="migration in available"
:key="migration.name"
class="ma-2"
>
<v-card-text>
<v-row align="center">
<v-col cols="12" sm="2">
<v-icon large color="primary"> mdi-import </v-icon>
</v-col>
<v-col cols="12" sm="10">
<div>
<strong>{{ migration.name }}</strong>
</div>
<div>{{ readableTime(migration.date) }}</div>
</v-col>
</v-row>
</v-card-text>
<v-card-actions class="mt-n6">
<v-spacer></v-spacer>
<v-btn color="error" text @click="deleteMigration(migration.name)">
Delete
</v-btn>
<v-btn color="accent" text @click="importMigration(migration.name)">
Import
</v-btn>
</v-card-actions>
</v-card>
</div>
<div v-else>
<v-card class="text-center ma-2">
<v-card-text>
No Migration Data Avaiable
</v-card-text>
</v-card>
</div>
<br />
</v-card>
</template>
<script>
import UploadBtn from "../../UI/UploadBtn";
import utils from "../../../utils";
import api from "../../../api";
export default {
props: {
folder: String,
title: String,
description: String,
available: Array,
},
components: {
UploadBtn,
},
data() {
return {
loading: false,
};
},
methods: {
deleteMigration(file_name) {
api.migrations.delete(this.folder, file_name);
this.$emit("refresh");
},
async importMigration(file_name) {
this.loading == true;
let response = await api.migrations.import(this.folder, file_name);
console.log(response);
this.$emit("imported", response.successful, response.failed);
this.loading == false;
},
readableTime(timestamp) {
let date = new Date(timestamp);
return utils.getDateAsText(date);
},
},
};
</script>
<style>
</style>

View file

@ -1,44 +1,96 @@
<template>
<div>
<SuccessFailureAlert
title="Migration Report"
ref="report"
failedHeader="Failed Imports"
:failed="failed"
successHeader="Successful Imports"
:success="success"
/>
<v-card :loading="loading">
<v-card-title class="headline"> {{$t('migration.recipe-migration')}} </v-card-title>
<v-card-title class="headline">
{{ $t("migration.recipe-migration") }}
</v-card-title>
<v-divider></v-divider>
<v-tabs v-model="tab">
<v-tab>Chowdown</v-tab>
<v-tab>Nextcloud Recipes</v-tab>
<v-tab-item>
<ChowdownCard @loading="loading = true" @finished="finished" />
</v-tab-item>
<v-tab-item>
<NextcloudCard @loading="loading = true" @finished="finished" />
</v-tab-item>
</v-tabs>
</v-card>
<v-row dense>
<v-col
:sm="6"
:md="6"
:lg="4"
:xl="3"
v-for="migration in migrations"
:key="migration.title"
>
<MigrationCard
:title="migration.title"
:folder="migration.urlVariable"
:description="migration.description"
:available="migration.availableImports"
@refresh="getAvailableMigrations"
@imported="showReport"
/>
</v-col>
</v-row>
</div>
</template>
<script>
import ChowdownCard from "./ChowdownCard";
import NextcloudCard from "./NextcloudCard";
// import SuccessFailureAlert from "../../UI/SuccessFailureAlert";
// import TimePicker from "./Webhooks/TimePicker";
import MigrationCard from "./MigrationCard";
import SuccessFailureAlert from "../../UI/SuccessFailureAlert";
import api from "../../../api";
export default {
components: {
ChowdownCard,
NextcloudCard,
MigrationCard,
SuccessFailureAlert,
},
data() {
return {
tab: null,
loading: false,
success: [],
failed: [],
migrations: {
nextcloud: {
title: "Nextcloud Cookbook",
description: "migrate data from a nextcloud cookbook intance",
urlVariable: "nextcloud",
availableImports: [],
},
chowdown: {
title: "Chowdown",
description: "Migrate From Chowdown",
urlVariable: "chowdown",
availableImports: [],
},
},
};
},
mounted() {
this.getAvailableMigrations();
},
methods: {
finished() {
this.loading = false;
this.$store.dispatch("requestRecentRecipes");
},
async getAvailableMigrations() {
let response = await api.migrations.getMigrations();
response.forEach(element => {
if (element.type === "nextcloud") {
this.migrations.nextcloud.availableImports = element.files;
} else if (element.type === "chowdown") {
this.migrations.chowdown.availableImports = element.files;
}
});
},
showReport(successful, failed) {
this.success = successful;
this.failed = failed;
this.$refs.report.open();
},
},
};
</script>

View file

@ -1,36 +1,28 @@
<template>
<div>
<v-btn block :color="value" @click="dialog = true">
{{ buttonText }}
</v-btn>
<v-dialog v-model="dialog" width="400">
<div class="text-center">
<h3>{{ buttonText }}</h3>
</div>
<v-text-field v-model="color" hide-details class="ma-0 pa-0" solo>
<template v-slot:append>
<v-menu
v-model="menu"
top
nudge-bottom="105"
nudge-left="16"
:close-on-content-click="false"
>
<template v-slot:activator="{ on }">
<div :style="swatchStyle" v-on="on" swatches-max-height="300" />
</template>
<v-card>
<v-card-title> {{ buttonText }} {{$t('settings.color')}} </v-card-title>
<v-card-text>
<v-text-field v-model="color"> </v-text-field>
<v-row>
<v-col></v-col>
<v-col>
<v-color-picker
dot-size="28"
hide-inputs
hide-mode-switch
mode="hexa"
:show-swatches="swatches"
swatches-max-height="300"
v-model="color"
@change="updateColor"
></v-color-picker>
</v-col>
<v-col></v-col>
</v-row>
<v-card-text class="pa-0">
<v-color-picker v-model="color" flat show-swatches />
</v-card-text>
<v-card-actions>
<v-btn text @click="toggleSwatches"> {{$t('settings.swatches')}} </v-btn>
<v-btn text @click="dialog = false"> {{$t('general.select')}} </v-btn>
</v-card-actions>
</v-card>
</v-dialog>
</v-menu>
</template>
</v-text-field>
</div>
</template>
@ -44,21 +36,30 @@ export default {
return {
dialog: false,
swatches: false,
color: "#FF00FF",
color: "#1976D2",
mask: "!#XXXXXXXX",
menu: false,
};
},
computed: {
swatchStyle() {
const { value, menu } = this;
return {
backgroundColor: value,
cursor: "pointer",
height: "30px",
width: "30px",
borderRadius: menu ? "50%" : "4px",
transition: "border-radius 200ms ease-in-out",
};
},
},
watch: {
color() {
this.updateColor();
},
},
methods: {
toggleSwatches() {
if (this.swatches) {
this.swatches = false;
} else this.swatches = true;
},
updateColor() {
this.$emit("input", this.color);
},

View file

@ -1,32 +1,50 @@
<template>
<div>
<v-alert v-if="success[0]" outlined dense type="success">
<v-dialog v-model="dialog" max-width="900px">
<v-card>
<v-card-title> {{ title }} </v-card-title>
<v-card-text class="mt-3">
<v-row>
<v-col>
<v-alert outlined dense type="success">
<h4>{{ successHeader }}</h4>
<v-list dense>
<v-list-item v-for="success in this.success" :key="success">
{{ success }}
</v-list-item>
</v-list>
<p v-for="success in this.success" :key="success" class="my-1">
- {{ success }}
</p>
</v-alert>
</v-col>
<v-col>
<v-alert v-if="failed[0]" outlined dense type="error">
<h4>{{ failedHeader }}</h4>
<v-list dense>
<v-list-item v-for="fail in this.failed" :key="fail">
{{ fail }}
</v-list-item>
</v-list>
<p v-for="fail in this.failed" :key="fail" class="my-1">
- {{ fail }}
</p>
</v-alert>
</div>
</v-col>
</v-row>
</v-card-text>
</v-card>
</v-dialog>
</template>
<script>
export default {
props: {
title: String,
successHeader: String,
success: Array,
failedHeader: String,
failed: Array,
},
data() {
return {
dialog: false,
};
},
methods: {
open() {
this.dialog = true;
},
},
};
</script>

View file

@ -1,23 +1,27 @@
import Vue from 'vue'
import VueI18n from 'vue-i18n'
import Vue from "vue";
import VueI18n from "vue-i18n";
Vue.use(VueI18n)
Vue.use(VueI18n);
function loadLocaleMessages() {
const locales = require.context('./locales', true, /[A-Za-z0-9-_,\s]+\.json$/i)
const messages = {}
const locales = require.context(
"./locales",
true,
/[A-Za-z0-9-_,\s]+\.json$/i
);
const messages = {};
locales.keys().forEach(key => {
const matched = key.match(/([A-Za-z0-9-_]+)\./i)
const matched = key.match(/([A-Za-z0-9-_]+)\./i);
if (matched && matched.length > 1) {
const locale = matched[1]
messages[locale] = locales(key)
const locale = matched[1];
messages[locale] = locales(key);
}
})
return messages
});
return messages;
}
export default new VueI18n({
locale: process.env.VUE_APP_I18N_LOCALE || 'en',
fallbackLocale: process.env.VUE_APP_I18N_FALLBACK_LOCALE || 'en',
messages: loadLocaleMessages()
})
locale: "en",
fallbackLocale: process.env.VUE_APP_I18N_FALLBACK_LOCALE || "en",
messages: loadLocaleMessages(),
});

View file

@ -4,7 +4,7 @@ import vuetify from "./plugins/vuetify";
import store from "./store/store";
import VueRouter from "vue-router";
import { routes } from "./routes";
import i18n from './i18n'
import i18n from "./i18n";
Vue.config.productionTip = false;
Vue.use(VueRouter);
@ -14,12 +14,13 @@ const router = new VueRouter({
mode: process.env.NODE_ENV === "production" ? "history" : "hash",
});
new Vue({
vuetify,
store,
router,
i18n,
render: (h) => h(App)
render: h => h(App),
}).$mount("#app");
// Truncate

View file

@ -13,7 +13,8 @@
"
>
</v-alert>
<Theme />
<General />
<Theme class="mt-2" />
<Backup class="mt-2" />
<Webhooks class="mt-2" />
<Migration class="mt-2" />
@ -39,6 +40,7 @@
<script>
import Backup from "../components/Settings/Backup";
import General from "../components/Settings/General";
import Webhooks from "../components/Settings/Webhook";
import Theme from "../components/Settings/Theme";
import Migration from "../components/Settings/Migration";
@ -50,6 +52,7 @@ export default {
Webhooks,
Theme,
Migration,
General,
},
data() {
return {

View file

@ -0,0 +1,44 @@
import VueI18n from "../../i18n";
const state = {
lang: "en",
allLangs: [
{
name: "English",
value: "en",
},
{
name: "Dutch",
value: "da",
},
{
name: "French",
value: "fr",
},
],
};
const mutations = {
setLang(state, payload) {
VueI18n.locale = payload;
state.lang = payload;
},
};
const actions = {
initLang({ getters }) {
VueI18n.locale = getters.getActiveLang;
},
};
const getters = {
getActiveLang: (state) => state.lang,
getAllLangs: (state) => state.allLangs,
};
export default {
state,
mutations,
actions,
getters,
};

View file

@ -3,17 +3,19 @@ import Vuex from "vuex";
import api from "../api";
import createPersistedState from "vuex-persistedstate";
import userSettings from "./modules/userSettings";
import language from "./modules/language";
Vue.use(Vuex);
const store = new Vuex.Store({
plugins: [
createPersistedState({
paths: ["userSettings"],
paths: ["userSettings", "language"],
}),
],
modules: {
userSettings,
language,
},
state: {
// Snackbar
@ -59,11 +61,11 @@ const store = new Vuex.Store({
getters: {
//
getSnackText: (state) => state.snackText,
getSnackActive: (state) => state.snackActive,
getSnackType: (state) => state.snackType,
getSnackText: state => state.snackText,
getSnackActive: state => state.snackActive,
getSnackType: state => state.snackType,
getRecentRecipes: (state) => state.recentRecipes,
getRecentRecipes: state => state.recentRecipes,
},
});

View file

@ -1,3 +1,6 @@
from datetime import datetime
from typing import List
from pydantic.main import BaseModel
@ -10,3 +13,13 @@ class ChowdownURL(BaseModel):
"url": "https://chowdownrepo.com/repo",
}
}
class MigrationFile(BaseModel):
name: str
date: datetime
class Migrations(BaseModel):
type: str
files: List[MigrationFile] = []

View file

@ -1,9 +1,11 @@
import operator
import shutil
from typing import List
from app_config import MIGRATION_DIR
from db.db_setup import generate_session
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
from models.migration_models import ChowdownURL
from models.migration_models import MigrationFile, Migrations
from services.migrations.chowdown import chowdown_migrate as chowdow_migrate
from services.migrations.nextcloud import migrate as nextcloud_migrate
from sqlalchemy.orm.session import Session
@ -12,51 +14,46 @@ from utils.snackbar import SnackResponse
router = APIRouter(tags=["Migration"])
# Chowdown
@router.post("/api/migration/chowdown/repo/")
def import_chowdown_recipes(repo: ChowdownURL, db: Session = Depends(generate_session)):
""" Import Chowsdown Recipes from Repo URL """
try:
report = chowdow_migrate(db, repo.url)
return SnackResponse.success(
"Recipes Imported from Git Repo, see report for failures.",
additional_data=report,
)
except:
return HTTPException(
status_code=400,
detail=SnackResponse.error(
"Unable to Migrate Recipes. See Log for Details"
),
)
# Nextcloud
@router.get("/api/migration/nextcloud/available/")
@router.get("/api/migrations/", response_model=List[Migrations])
def get_avaiable_nextcloud_imports():
""" Returns a list of avaiable directories that can be imported into Mealie """
available = []
for dir in MIGRATION_DIR.iterdir():
if dir.is_dir():
available.append(dir.stem)
elif dir.suffix == ".zip":
available.append(dir.name)
response_data = []
migration_dirs = [
MIGRATION_DIR.joinpath("nextcloud"),
MIGRATION_DIR.joinpath("chowdown"),
]
for directory in migration_dirs:
migration = Migrations(type=directory.stem)
for zip in directory.iterdir():
if zip.suffix == ".zip":
migration_zip = MigrationFile(name=zip.name, date=zip.stat().st_ctime)
migration.files.append(migration_zip)
response_data.append(migration)
return available
migration.files.sort(key=operator.attrgetter("date"), reverse=True)
return response_data
@router.post("/api/migration/nextcloud/{selection}/import/")
def import_nextcloud_directory(selection: str, db: Session = Depends(generate_session)):
@router.post("/api/migrations/{type}/{file_name}/import/")
def import_nextcloud_directory(
type: str, file_name: str, db: Session = Depends(generate_session)
):
""" Imports all the recipes in a given directory """
return nextcloud_migrate(db, selection)
file_path = MIGRATION_DIR.joinpath(type, file_name)
if type == "nextcloud":
return nextcloud_migrate(db, file_path)
elif type == "chowdown":
return chowdow_migrate(db, file_path)
else:
return SnackResponse.error("Incorrect Migration Type Selected")
@router.delete("/api/migration/{file_folder_name}/delete/")
def delete_migration_data(file_folder_name: str):
@router.delete("/api/migrations/{folder}/{file}/delete/")
def delete_migration_data(folder: str, file: str):
""" Removes migration data from the file system """
remove_path = MIGRATION_DIR.joinpath(file_folder_name)
remove_path = MIGRATION_DIR.joinpath(folder, file)
if remove_path.is_file():
remove_path.unlink()
@ -68,10 +65,12 @@ def delete_migration_data(file_folder_name: str):
return SnackResponse.info(f"Migration Data Remove: {remove_path.absolute()}")
@router.post("/api/migration/upload/")
def upload_nextcloud_zipfile(archive: UploadFile = File(...)):
@router.post("/api/migrations/{type}/upload/")
def upload_nextcloud_zipfile(type: str, archive: UploadFile = File(...)):
""" Upload a .zip File to later be imported into Mealie """
dest = MIGRATION_DIR.joinpath(archive.filename)
dir = MIGRATION_DIR.joinpath(type)
dir.mkdir(parents=True, exist_ok=True)
dest = dir.joinpath(archive.filename)
with dest.open("wb") as buffer:
shutil.copyfileobj(archive.file, buffer)

View file

@ -1,11 +1,11 @@
import shutil
from pathlib import Path
import git
import yaml
from app_config import IMG_DIR, TEMP_DIR
from services.recipe_services import Recipe
from sqlalchemy.orm.session import Session
from utils.unzip import unpack_zip
try:
from yaml import CLoader as Loader
@ -13,20 +13,6 @@ except ImportError:
from yaml import Loader
def pull_repo(repo):
dest_dir = TEMP_DIR.joinpath("/migration/git_pull")
if dest_dir.exists():
shutil.rmtree(dest_dir)
dest_dir.mkdir(parents=True, exist_ok=True)
git.Git(dest_dir).clone(repo)
repo_name = repo.split("/")[-1]
recipe_dir = dest_dir.joinpath(repo_name, "_recipes")
image_dir = dest_dir.joinpath(repo_name, "images")
return recipe_dir, image_dir
def read_chowdown_file(recipe_file: Path) -> Recipe:
"""Parse through the yaml file to try and pull out the relavent information.
Some issues occur when ":" are used in the text. I have no put a lot of effort
@ -74,25 +60,31 @@ def read_chowdown_file(recipe_file: Path) -> Recipe:
return new_recipe
def chowdown_migrate(session: Session, repo):
recipe_dir, image_dir = pull_repo(repo)
def chowdown_migrate(session: Session, zip_file: Path):
temp_dir = unpack_zip(zip_file)
failed_images = []
for image in image_dir.iterdir():
try:
shutil.copy(image, IMG_DIR.joinpath(image.name))
except:
failed_images.append(image.name)
with temp_dir as dir:
image_dir = TEMP_DIR.joinpath(dir, zip_file.stem, "images")
recipe_dir = TEMP_DIR.joinpath(dir, zip_file.stem, "_recipes")
failed_recipes = []
successful_recipes = []
for recipe in recipe_dir.glob("*.md"):
try:
new_recipe = read_chowdown_file(recipe)
new_recipe.save_to_db(session)
successful_recipes.append(recipe.stem)
except:
failed_recipes.append(recipe.name)
failed_recipes.append(recipe.stem)
report = {"failedImages": failed_images, "failedRecipes": failed_recipes}
failed_images = []
for image in image_dir.iterdir():
try:
if not image.stem in failed_recipes:
shutil.copy(image, IMG_DIR.joinpath(image.name))
except:
failed_images.append(image.name)
report = {"successful": successful_recipes, "failed": failed_recipes}
return report

View file

@ -35,14 +35,14 @@ class Recipe(BaseModel):
# Mealie Specific
slug: Optional[str] = ""
categories: Optional[List[str]]
tags: Optional[List[str]]
categories: Optional[List[str]] = []
tags: Optional[List[str]] = []
dateAdded: Optional[datetime.date]
notes: Optional[List[RecipeNote]]
notes: Optional[List[RecipeNote]] = []
rating: Optional[int]
rating: Optional[int]
orgURL: Optional[str]
extras: Optional[dict]
extras: Optional[dict] = {}
class Config:
schema_extra = {

View file

@ -36,17 +36,17 @@ def nextcloud_zip():
def test_upload_nextcloud_zip(api_client, nextcloud_zip):
response = api_client.post(
"/api/migration/upload/", files={"archive": nextcloud_zip.open("rb")}
"/api/migrations/nextcloud/upload/", files={"archive": nextcloud_zip.open("rb")}
)
assert response.status_code == 200
assert MIGRATION_DIR.joinpath(nextcloud_zip.name).is_file()
assert MIGRATION_DIR.joinpath("nextcloud", nextcloud_zip.name).is_file()
def test_import_nextcloud_directory(api_client, nextcloud_zip):
selection = nextcloud_zip.name
response = api_client.post(f"/api/migration/nextcloud/{selection}/import/")
response = api_client.post(f"/api/migrations/nextcloud/{selection}/import/")
assert response.status_code == 200
@ -60,7 +60,7 @@ def test_import_nextcloud_directory(api_client, nextcloud_zip):
def test_delete_migration_data(api_client, nextcloud_zip):
selection = nextcloud_zip.name
response = api_client.delete(f"/api/migration/{selection}/delete/")
response = api_client.delete(f"/api/migrations/nextcloud/{selection}/delete/")
assert response.status_code == 200
assert not MIGRATION_DIR.joinpath(nextcloud_zip.name).is_file()

18
mealie/utils/unzip.py Normal file
View file

@ -0,0 +1,18 @@
import tempfile
import zipfile
from pathlib import Path
from app_config import TEMP_DIR
def unpack_zip(selection: Path) -> tempfile.TemporaryDirectory:
temp_dir = tempfile.TemporaryDirectory(dir=TEMP_DIR)
temp_dir_path = Path(temp_dir.name)
if selection.suffix == ".zip":
with zipfile.ZipFile(selection, "r") as zip_ref:
zip_ref.extractall(path=temp_dir_path)
else:
raise Exception("File is not a zip file")
return temp_dir

44
poetry.lock generated
View file

@ -211,28 +211,6 @@ dev = ["python-jose[cryptography] (>=3.1.0,<4.0.0)", "passlib[bcrypt] (>=1.7.2,<
doc = ["mkdocs (>=1.1.2,<2.0.0)", "mkdocs-material (>=6.1.4,<7.0.0)", "markdown-include (>=0.5.1,<0.6.0)", "mkdocs-markdownextradata-plugin (>=0.1.7,<0.2.0)", "typer-cli (>=0.0.9,<0.0.10)", "pyyaml (>=5.3.1,<6.0.0)"]
test = ["pytest (==5.4.3)", "pytest-cov (==2.10.0)", "pytest-asyncio (>=0.14.0,<0.15.0)", "mypy (==0.790)", "flake8 (>=3.8.3,<4.0.0)", "black (==20.8b1)", "isort (>=5.0.6,<6.0.0)", "requests (>=2.24.0,<3.0.0)", "httpx (>=0.14.0,<0.15.0)", "email_validator (>=1.1.1,<2.0.0)", "sqlalchemy (>=1.3.18,<2.0.0)", "peewee (>=3.13.3,<4.0.0)", "databases[sqlite] (>=0.3.2,<0.4.0)", "orjson (>=3.2.1,<4.0.0)", "async_exit_stack (>=1.0.1,<2.0.0)", "async_generator (>=1.10,<2.0.0)", "python-multipart (>=0.0.5,<0.0.6)", "aiofiles (>=0.5.0,<0.6.0)", "flask (>=1.1.2,<2.0.0)"]
[[package]]
name = "gitdb"
version = "4.0.5"
description = "Git Object Database"
category = "main"
optional = false
python-versions = ">=3.4"
[package.dependencies]
smmap = ">=3.0.1,<4"
[[package]]
name = "gitpython"
version = "3.1.12"
description = "Python Git Library"
category = "main"
optional = false
python-versions = ">=3.4"
[package.dependencies]
gitdb = ">=4.0.1,<5"
[[package]]
name = "h11"
version = "0.12.0"
@ -667,14 +645,6 @@ category = "main"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*"
[[package]]
name = "smmap"
version = "3.0.4"
description = "A pure Python implementation of a sliding window memory map manager"
category = "main"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
[[package]]
name = "soupsieve"
version = "2.1"
@ -861,7 +831,7 @@ python-versions = "*"
[metadata]
lock-version = "1.1"
python-versions = "^3.8"
content-hash = "115f5c8741bf1820e1b1ee8175616ea9f50f906803eac43a24bf259a5507e5d1"
content-hash = "fbe2a3d2885fcc24abe5a285a961f968ea32aa22182f88f44a7c9dd624c968b5"
[metadata.files]
aiofiles = [
@ -979,14 +949,6 @@ fastapi = [
{file = "fastapi-0.63.0-py3-none-any.whl", hash = "sha256:98d8ea9591d8512fdadf255d2a8fa56515cdd8624dca4af369da73727409508e"},
{file = "fastapi-0.63.0.tar.gz", hash = "sha256:63c4592f5ef3edf30afa9a44fa7c6b7ccb20e0d3f68cd9eba07b44d552058dcb"},
]
gitdb = [
{file = "gitdb-4.0.5-py3-none-any.whl", hash = "sha256:91f36bfb1ab7949b3b40e23736db18231bf7593edada2ba5c3a174a7b23657ac"},
{file = "gitdb-4.0.5.tar.gz", hash = "sha256:c9e1f2d0db7ddb9a704c2a0217be31214e91a4fe1dea1efad19ae42ba0c285c9"},
]
gitpython = [
{file = "GitPython-3.1.12-py3-none-any.whl", hash = "sha256:867ec3dfb126aac0f8296b19fb63b8c4a399f32b4b6fafe84c4b10af5fa9f7b5"},
{file = "GitPython-3.1.12.tar.gz", hash = "sha256:42dbefd8d9e2576c496ed0059f3103dcef7125b9ce16f9d5f9c834aed44a1dac"},
]
h11 = [
{file = "h11-0.12.0-py3-none-any.whl", hash = "sha256:36a3cb8c0a032f56e2da7084577878a035d3b61d104230d4bd49c0c6b555a9c6"},
{file = "h11-0.12.0.tar.gz", hash = "sha256:47222cb6067e4a307d535814917cd98fd0a57b6788ce715755fa2b6c28b56042"},
@ -1307,10 +1269,6 @@ six = [
{file = "six-1.15.0-py2.py3-none-any.whl", hash = "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"},
{file = "six-1.15.0.tar.gz", hash = "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259"},
]
smmap = [
{file = "smmap-3.0.4-py2.py3-none-any.whl", hash = "sha256:54c44c197c819d5ef1991799a7e30b662d1e520f2ac75c9efbeb54a742214cf4"},
{file = "smmap-3.0.4.tar.gz", hash = "sha256:9c98bbd1f9786d22f14b3d4126894d56befb835ec90cef151af566c7e19b5d24"},
]
soupsieve = [
{file = "soupsieve-2.1-py3-none-any.whl", hash = "sha256:4bb21a6ee4707bf43b61230e80740e71bfe56e55d1f1f50924b087bb2975c851"},
{file = "soupsieve-2.1.tar.gz", hash = "sha256:6dc52924dc0bc710a5d16794e6b3480b2c7c08b07729505feab2b2c16661ff6e"},

View file

@ -15,7 +15,6 @@ aniso8601 = "7.0.0"
appdirs = "1.4.4"
fastapi = "^0.63.0"
uvicorn = {extras = ["standard"], version = "^0.13.0"}
GitPython = "^3.1.12"
APScheduler = "^3.6.3"
SQLAlchemy = "^1.3.22"
Jinja2 = "^2.11.2"